2023-10-06T02:32:32.597 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T02:32:32.597 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} --verbose --archive /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156 --name yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/orig.config.yaml 2023-10-06T02:32:32.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T02:32:32.732 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156 branch: wip-yuri-testing-2023-10-05-0730-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} email: yweinste@redhat.com first_in_suite: false job_id: '7412156' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri-testing-2023-10-05-0730-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-05-0730-quincy sha1: f1e315a9732d36c05ec316678e2397dc58561c7f owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7519 sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-05-0730-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f1e315a9732d36c05ec316678e2397dc58561c7f targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:52:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T02:32:32.733 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa; will attempt to use it 2023-10-06T02:32:32.734 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks 2023-10-06T02:32:32.734 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T02:32:32.736 INFO:teuthology.task.internal:Checking packages... 2023-10-06T02:32:32.764 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2' 2023-10-06T02:32:32.764 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T02:32:32.765 INFO:teuthology.packaging:ref: None 2023-10-06T02:32:32.765 INFO:teuthology.packaging:tag: None 2023-10-06T02:32:32.765 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T02:32:32.765 INFO:teuthology.packaging:sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:32:32.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T02:32:32.905 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1296-gf1e315a9-1focal 2023-10-06T02:32:32.905 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T02:32:32.912 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T02:32:32.913 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T02:32:32.943 INFO:teuthology.task.internal:Saving configuration 2023-10-06T02:32:32.964 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T02:32:32.977 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:32:33.007 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:26:16.714597', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-06T02:32:33.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:26:16.713100', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-06T02:32:33.034 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T02:32:33.040 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T02:32:33.040 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T02:32:33.040 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T02:32:33.146 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f86fa402040>, signals=[15]) 2023-10-06T02:32:33.146 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T02:32:33.152 INFO:teuthology.task.internal:Opening connections... 2023-10-06T02:32:33.152 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:32:33.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:32:33.202 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:32:33.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:32:33.248 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T02:32:33.255 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2023-10-06T02:32:33.285 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2023-10-06T02:32:33.286 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Ubuntu" 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:ID=ubuntu 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE=debian 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="20.04" 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-06T02:32:33.334 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-06T02:32:33.335 INFO:teuthology.orchestra.run.smithi033.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-06T02:32:33.335 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_CODENAME=focal 2023-10-06T02:32:33.335 INFO:teuthology.orchestra.run.smithi033.stdout:UBUNTU_CODENAME=focal 2023-10-06T02:32:33.335 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-10-06T02:32:33.358 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-10-06T02:32:33.364 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-10-06T02:32:33.364 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-10-06T02:32:33.413 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Ubuntu" 2023-10-06T02:32:33.413 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:ID=ubuntu 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE=debian 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="20.04" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_CODENAME=focal 2023-10-06T02:32:33.414 INFO:teuthology.orchestra.run.smithi167.stdout:UBUNTU_CODENAME=focal 2023-10-06T02:32:33.415 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-10-06T02:32:33.432 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T02:32:33.441 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T02:32:33.462 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T02:32:33.462 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2023-10-06T02:32:33.465 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2023-10-06T02:32:33.470 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T02:32:33.508 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T02:32:33.509 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2023-10-06T02:32:33.511 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2023-10-06T02:32:33.526 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T02:32:33.583 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T02:32:33.598 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T02:32:33.598 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T02:32:33.598 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T02:32:33.599 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T02:32:33.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:32:33.604 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2023-10-06T02:32:33.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:32:33.605 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-10-06T02:32:33.611 INFO:teuthology.orchestra.run.smithi033.stdout:5.4.0-124-generic 2023-10-06T02:32:33.611 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.4.0-124-generic 2023-10-06T02:32:33.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2023-10-06T02:32:33.612 INFO:teuthology.orchestra.run.smithi167.stdout:5.4.0-124-generic 2023-10-06T02:32:33.612 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.4.0-124-generic 2023-10-06T02:32:33.613 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2023-10-06T02:32:33.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2023-10-06T02:32:33.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2023-10-06T02:32:33.903 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T02:32:33.904 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-06T02:32:33.904 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-06T02:32:33.972 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-06T02:32:34.017 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T02:32:34.027 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-06T02:32:34.075 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-06T02:32:34.294 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-06T02:32:34.419 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-06T02:32:34.429 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-06T02:32:34.435 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,888 kB] 2023-10-06T02:32:34.441 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-06T02:32:34.444 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-06T02:32:34.445 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-06T02:32:34.445 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-06T02:32:34.463 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-06T02:32:34.466 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T02:32:34.466 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-06T02:32:34.470 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-06T02:32:34.476 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-06T02:32:34.478 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-06T02:32:34.483 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-06T02:32:34.483 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-06T02:32:34.483 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-06T02:32:34.502 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-06T02:32:34.523 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-06T02:32:34.534 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-06T02:32:34.544 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-06T02:32:34.549 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-06T02:32:34.549 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-06T02:32:34.564 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-06T02:32:34.565 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-06T02:32:34.567 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T02:32:34.567 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-06T02:32:34.575 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-06T02:32:34.579 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-06T02:32:34.581 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-06T02:32:34.581 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-06T02:32:34.582 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-06T02:32:34.596 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-06T02:32:34.598 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-06T02:32:34.598 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-06T02:32:34.598 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-06T02:32:34.598 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-06T02:32:34.612 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-06T02:32:34.613 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-06T02:32:34.613 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-06T02:32:34.666 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,888 kB] 2023-10-06T02:32:35.017 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-06T02:32:35.031 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-06T02:32:35.040 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-06T02:32:35.041 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-06T02:32:35.116 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-06T02:32:35.116 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-06T02:32:35.123 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T02:32:35.123 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-06T02:32:35.130 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-06T02:32:35.137 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-06T02:32:35.147 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-06T02:32:35.156 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-06T02:32:35.157 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-06T02:32:35.157 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-06T02:32:35.233 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-06T02:32:35.240 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-06T02:32:35.320 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-06T02:32:35.403 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-06T02:32:35.736 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-06T02:32:35.742 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-06T02:32:35.744 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-06T02:32:35.752 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-06T02:32:35.754 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-06T02:32:36.037 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-06T02:32:36.075 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-06T02:32:36.088 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-06T02:32:36.090 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-06T02:32:36.091 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-06T02:32:36.092 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-06T02:32:36.092 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-06T02:32:36.092 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-06T02:32:36.117 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-06T02:32:36.120 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-06T02:32:36.284 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-06T02:32:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-06T02:32:36.489 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-06T02:32:38.250 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 17.6 MB in 3s (6,930 kB/s) 2023-10-06T02:32:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 17.6 MB in 3s (5,530 kB/s) 2023-10-06T02:32:39.714 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:32:39.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T02:32:39.811 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:32:40.009 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2023-10-06T02:32:40.010 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2023-10-06T02:32:40.184 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2023-10-06T02:32:40.184 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T02:32:40.184 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-06T02:32:40.185 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-06T02:32:40.185 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2023-10-06T02:32:40.186 INFO:teuthology.orchestra.run.smithi167.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-06T02:32:40.186 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2023-10-06T02:32:40.186 INFO:teuthology.orchestra.run.smithi167.stdout: thermald 2023-10-06T02:32:40.239 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2023-10-06T02:32:40.240 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T02:32:40.240 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-06T02:32:40.240 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-06T02:32:40.241 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2023-10-06T02:32:40.241 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-06T02:32:40.300 INFO:teuthology.orchestra.run.smithi167.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T02:32:40.300 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 77.0 MB of archives. 2023-10-06T02:32:40.300 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-06T02:32:40.300 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-06T02:32:40.458 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:32:40.483 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T02:32:40.561 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:32:40.701 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-06T02:32:40.746 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2023-10-06T02:32:40.747 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2023-10-06T02:32:40.817 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-06T02:32:40.930 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2023-10-06T02:32:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T02:32:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-06T02:32:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-06T02:32:40.931 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2023-10-06T02:32:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-06T02:32:40.931 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2023-10-06T02:32:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: thermald 2023-10-06T02:32:40.989 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2023-10-06T02:32:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-06T02:32:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-06T02:32:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-06T02:32:40.990 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2023-10-06T02:32:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-06T02:32:41.070 INFO:teuthology.orchestra.run.smithi033.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T02:32:41.070 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 77.0 MB of archives. 2023-10-06T02:32:41.071 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-06T02:32:41.071 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-06T02:32:41.426 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-06T02:32:41.426 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-06T02:32:41.427 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-06T02:32:41.455 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-06T02:32:41.594 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-06T02:32:41.613 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-06T02:32:41.622 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-06T02:32:42.191 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-06T02:32:42.191 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-06T02:32:42.191 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-06T02:32:42.284 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-06T02:32:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-06T02:32:42.317 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 77.0 MB in 1s (56.4 MB/s) 2023-10-06T02:32:42.537 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-06T02:32:43.018 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 77.0 MB in 1s (59.3 MB/s) 2023-10-06T02:32:43.206 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-06T02:32:44.257 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-06T02:32:44.262 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:44.364 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:44.882 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-06T02:32:44.887 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:45.028 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:46.348 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-06T02:32:46.361 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:46.497 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:46.988 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-06T02:32:47.002 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:47.106 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:47.623 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-06T02:32:47.639 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:47.685 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:48.241 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-06T02:32:48.257 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:48.299 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:52.593 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T02:32:52.718 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T02:32:53.085 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T02:32:53.182 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T02:32:53.230 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T02:32:53.324 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T02:32:53.575 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-06T02:32:53.587 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-06T02:32:53.624 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T02:32:53.683 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T02:32:53.802 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T02:32:54.180 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-06T02:32:54.193 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-06T02:32:54.230 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T02:32:59.082 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-06T02:32:59.104 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:32:59.149 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:32:59.997 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-06T02:33:00.019 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-06T02:33:00.063 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:03.120 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T02:33:03.161 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-06T02:33:03.217 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T02:33:03.283 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-06T02:33:03.564 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T02:33:03.642 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-06T02:33:03.714 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:03.771 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:03.840 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:03.930 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:05.116 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:05.149 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:06.292 INFO:teuthology.orchestra.run.smithi167.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-06T02:33:06.293 INFO:teuthology.orchestra.run.smithi167.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-06T02:33:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:06.426 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-06T02:33:06.501 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-06T02:33:06.569 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:06.628 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:07.854 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-06T02:33:07.997 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-06T02:33:08.132 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:08.216 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-06T02:33:08.216 INFO:teuthology.orchestra.run.smithi033.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:08.279 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-06T02:33:08.413 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-06T02:33:08.540 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-06T02:33:08.636 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-06T02:33:08.636 INFO:teuthology.orchestra.run.smithi167.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:30.592 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-06T02:33:30.592 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub' 2023-10-06T02:33:30.615 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T02:33:30.629 INFO:teuthology.orchestra.run.smithi033.stdout:Generating grub configuration file ... 2023-10-06T02:33:30.931 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T02:33:30.952 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:31.036 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-06T02:33:31.036 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub' 2023-10-06T02:33:31.049 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T02:33:31.069 INFO:teuthology.orchestra.run.smithi167.stdout:Generating grub configuration file ... 2023-10-06T02:33:31.382 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T02:33:31.393 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T02:33:31.449 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T02:33:31.478 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:31.916 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T02:33:31.927 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T02:33:33.742 INFO:teuthology.orchestra.run.smithi167.stdout:done 2023-10-06T02:33:34.146 INFO:teuthology.orchestra.run.smithi033.stdout:done 2023-10-06T02:33:34.202 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 20 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T02:33:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.4.0.164.161 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2023-10-06T02:33:34.245 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2023-10-06T02:33:34.246 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-06T02:33:34.246 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T02:33:34.246 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T02:33:34.246 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T02:33:34.247 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T02:33:34.247 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2023-10-06T02:33:34.333 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2023-10-06T02:33:34.468 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T02:33:34.475 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T02:33:34.492 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T02:33:34.583 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2023-10-06T02:33:34.606 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T02:33:34.635 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2023-10-06T02:33:34.635 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 20 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.4.0.164.161 2023-10-06T02:33:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T02:33:34.637 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T02:33:34.637 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2023-10-06T02:33:34.637 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2023-10-06T02:33:34.637 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2023-10-06T02:33:34.637 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2023-10-06T02:33:34.637 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-06T02:33:34.637 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T02:33:34.637 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T02:33:34.638 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T02:33:34.638 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T02:33:34.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2023-10-06T02:33:34.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2023-10-06T02:33:34.866 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T02:33:34.875 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T02:33:34.895 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T02:33:34.980 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T02:33:36.217 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:33:36.242 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T02:33:36.309 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:33:36.508 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2023-10-06T02:33:36.509 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2023-10-06T02:33:36.681 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:33:36.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-06T02:33:36.714 INFO:teuthology.orchestra.run.smithi167.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-06T02:33:36.714 INFO:teuthology.orchestra.run.smithi167.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T02:33:36.716 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2023-10-06T02:33:36.736 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2023-10-06T02:33:36.736 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2023-10-06T02:33:36.736 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 20 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.4.0.164.161 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2023-10-06T02:33:36.737 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2023-10-06T02:33:36.738 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2023-10-06T02:33:36.738 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T02:33:36.772 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:33:36.786 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.lHTJDVNkYp 2023-10-06T02:33:36.787 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub/grub.cfg /tmp/tmp.lHTJDVNkYp 2023-10-06T02:33:36.804 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.lHTJDVNkYp 2023-10-06T02:33:36.888 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.lHTJDVNkYp is 9KB 2023-10-06T02:33:36.903 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.lHTJDVNkYp 2023-10-06T02:33:36.909 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-06T02:33:36.967 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:33:36.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-06T02:33:36.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-06T02:33:36.974 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2023-10-06T02:33:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2023-10-06T02:33:37.000 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-06T02:33:37.001 DEBUG:teuthology.orchestra.run.smithi167:> sudo update-grub 2023-10-06T02:33:37.205 INFO:teuthology.orchestra.run.smithi033.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-06T02:33:37.205 INFO:teuthology.orchestra.run.smithi033.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-06T02:33:37.207 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2023-10-06T02:33:37.227 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 20 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.4.0.164.161 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-06T02:33:37.228 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-06T02:33:37.229 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2023-10-06T02:33:37.229 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2023-10-06T02:33:37.229 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2023-10-06T02:33:37.229 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2023-10-06T02:33:37.229 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2023-10-06T02:33:37.233 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.OtedRogML6 2023-10-06T02:33:37.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub/grub.cfg /tmp/tmp.OtedRogML6 2023-10-06T02:33:37.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.OtedRogML6 2023-10-06T02:33:37.283 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.OtedRogML6 is 9KB 2023-10-06T02:33:37.297 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.OtedRogML6 2023-10-06T02:33:37.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-06T02:33:37.359 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:33:37.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-06T02:33:37.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-06T02:33:37.389 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-06T02:33:37.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo update-grub 2023-10-06T02:33:38.333 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub' 2023-10-06T02:33:38.343 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T02:33:38.385 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2023-10-06T02:33:38.665 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T02:33:38.677 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:38.786 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub' 2023-10-06T02:33:38.796 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T02:33:38.809 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2023-10-06T02:33:39.074 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-06T02:33:39.086 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-06T02:33:39.126 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T02:33:39.138 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T02:33:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T02:33:39.464 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T02:33:39.938 INFO:teuthology.orchestra.run.smithi167.stderr:done 2023-10-06T02:33:39.940 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2023-10-06T02:33:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:done 2023-10-06T02:33:41.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2023-10-06T02:34:09.970 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T02:34:09.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:34:09.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:34:11.133 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T02:34:11.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:34:11.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:34:28.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-06T02:34:29.626 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-10-06T02:34:37.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:34:37.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:34:38.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:34:38.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:34:40.410 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-06T02:34:52.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:34:52.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:34:57.018 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-10-06T02:35:09.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:35:09.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:35:12.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-10-06T02:35:23.898 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-06T02:35:27.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:35:27.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:35:27.324 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-10-06T02:35:27.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:35:27.821 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-164-generic"... 2023-10-06T02:35:27.822 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2023-10-06T02:35:27.828 INFO:teuthology.orchestra.run.smithi033.stdout:5.4.0-164-generic 2023-10-06T02:35:27.828 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-06T02:35:27.828 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T02:35:27.829 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-06T02:35:28.829 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T02:35:28.829 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T02:35:28.916 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2023-10-06T02:35:28.940 DEBUG:teuthology.parallel:result is None 2023-10-06T02:35:38.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:35:38.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:35:39.110 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-06T02:35:39.700 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:35:39.700 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-164-generic"... 2023-10-06T02:35:39.700 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-10-06T02:35:39.706 INFO:teuthology.orchestra.run.smithi167.stdout:5.4.0-164-generic 2023-10-06T02:35:39.706 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-06T02:35:39.707 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T02:35:39.707 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-06T02:35:40.707 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T02:35:40.708 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T02:35:40.801 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2023-10-06T02:35:40.838 DEBUG:teuthology.parallel:result is None 2023-10-06T02:35:40.839 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T02:35:40.846 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T02:35:40.846 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T02:35:40.848 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T02:35:40.855 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T02:35:40.863 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T02:35:40.874 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T02:35:40.874 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T02:35:40.876 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T02:35:40.898 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T02:35:40.926 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T02:35:40.927 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T02:35:40.936 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T02:35:40.952 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:35:40.954 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:35:40.962 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:35:40.964 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:35:40.966 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T02:35:41.030 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T02:35:41.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T02:35:41.033 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T02:35:41.049 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T02:35:41.060 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T02:35:41.061 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T02:35:41.091 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T02:35:41.098 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:35:41.104 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:35:41.110 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:35:41.110 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T02:35:41.169 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:35:41.176 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:35:41.183 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:35:41.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T02:35:41.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2023-10-06T02:35:41.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2023-10-06T02:35:41.310 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T02:35:41.317 INFO:teuthology.task.internal:Starting timer... 2023-10-06T02:35:41.318 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T02:35:41.355 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T02:35:41.392 DEBUG:teuthology.task.selinux:Excluding smithi033: OS 'ubuntu' does not support SELinux 2023-10-06T02:35:41.392 DEBUG:teuthology.task.selinux:Excluding smithi167: OS 'ubuntu' does not support SELinux 2023-10-06T02:35:41.392 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T02:35:41.393 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-06T02:35:41.393 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T02:35:41.393 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T02:35:41.419 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T02:35:41.420 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T02:35:41.435 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T02:35:41.436 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T02:41:23.903 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2023-10-06T02:41:23.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:41:23.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:41:23.956 DEBUG:teuthology.orchestra.run.smithi033:> true 2023-10-06T02:41:24.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2023-10-06T02:41:24.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:41:24.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:41:24.122 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-06T02:41:24.244 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T02:41:24.245 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T02:41:24.261 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T02:41:24.261 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T02:41:24.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:41:24.267 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T02:41:24.267 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:41:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-06T02:41:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Command line: ntpd -gq 2023-10-06T02:41:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: proto: precision = 0.105 usec (-23) 2023-10-06T02:41:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T02:41:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T02:41:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: restrict ::: KOD does nothing without LIMITED. 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listen and drop on 0 v6wildcard [::]:123 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listen normally on 2 lo 127.0.0.1:123 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listen normally on 3 ens1f0 172.21.15.33:123 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listen normally on 4 lo [::1]:123 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:d294%4]:123 2023-10-06T02:41:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:24 ntpd[22783]: Listening on routing socket on fd #22 for interface updates 2023-10-06T02:41:24.304 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-06T02:41:24.304 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Command line: ntpd -gq 2023-10-06T02:41:24.304 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: proto: precision = 0.078 usec (-24) 2023-10-06T02:41:24.305 INFO:teuthology.orchestra.run.smithi167.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T02:41:24.305 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T02:41:24.305 INFO:teuthology.orchestra.run.smithi167.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-06T02:41:24.306 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: restrict ::: KOD does nothing without LIMITED. 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listen and drop on 0 v6wildcard [::]:123 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listen normally on 2 lo 127.0.0.1:123 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listen normally on 3 enp3s0f1 172.21.15.167:123 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listen normally on 4 lo [::1]:123 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7309%5]:123 2023-10-06T02:41:24.307 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:24 ntpd[22850]: Listening on routing socket on fd #22 for interface updates 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:25.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:25.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:25.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:25 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:25.304 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:25.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:25 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:27.301 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:27.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:27 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:27.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:27.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:27 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:28.304 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:28 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:28.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:28 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:28.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:28 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:28.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:28 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:28.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:28 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:28.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:28 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:29.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:29.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:29 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:29.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:29.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:29.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:29 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:30.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:30 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:30.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:30 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:30.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:30 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:30.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:30 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:30.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:30 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:30.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:30 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:31.301 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.301 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.301 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.301 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stdout: 6 Oct 02:41:31 ntpd[22783]: ntpd: time slew +0.000275 s 2023-10-06T02:41:31.302 INFO:teuthology.orchestra.run.smithi033.stdout:ntpd: time slew +0.000275s 2023-10-06T02:41:31.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-06T02:41:31.303 INFO:teuthology.orchestra.run.smithi033.stderr: 6 Oct 02:41:31 ntpd[22783]: can't open /var/log/ntpstats/loopstats.20231006: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T02:41:31.305 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T02:41:31.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T02:41:31.306 INFO:teuthology.orchestra.run.smithi167.stdout: 6 Oct 02:41:31 ntpd[22850]: ntpd: time slew +0.001769 s 2023-10-06T02:41:31.306 INFO:teuthology.orchestra.run.smithi167.stdout:ntpd: time slew +0.001769s 2023-10-06T02:41:31.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-06T02:41:31.306 INFO:teuthology.orchestra.run.smithi167.stderr: 6 Oct 02:41:31 ntpd[22850]: can't open /var/log/ntpstats/loopstats.20231006: Permission denied 2023-10-06T02:41:31.367 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T02:41:31.368 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2023-10-06T02:41:31.368 INFO:teuthology.orchestra.run.smithi033.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.368 INFO:teuthology.orchestra.run.smithi033.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.368 INFO:teuthology.orchestra.run.smithi033.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.368 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.369 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T02:41:31.370 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2023-10-06T02:41:31.370 INFO:teuthology.orchestra.run.smithi167.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.370 INFO:teuthology.orchestra.run.smithi167.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.370 INFO:teuthology.orchestra.run.smithi167.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.370 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:41:31.370 INFO:teuthology.run_tasks:Running task install... 2023-10-06T02:41:31.378 DEBUG:teuthology.task.install:project ceph 2023-10-06T02:41:31.378 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}} 2023-10-06T02:41:31.379 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'} 2023-10-06T02:41:31.379 INFO:teuthology.task.install:Using flavor: default 2023-10-06T02:41:31.384 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T02:41:31.385 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T02:41:31.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-key list | grep Ceph 2023-10-06T02:41:31.386 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-key list | grep Ceph 2023-10-06T02:41:31.486 INFO:teuthology.orchestra.run.smithi033.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-06T02:41:31.509 INFO:teuthology.orchestra.run.smithi167.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-06T02:41:31.551 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-06T02:41:31.551 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph.com (release key) 2023-10-06T02:41:31.552 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-06T02:41:31.552 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-06T02:41:31.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:41:31.570 INFO:teuthology.orchestra.run.smithi167.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-06T02:41:31.570 INFO:teuthology.orchestra.run.smithi167.stdout:uid [ unknown] Ceph.com (release key) 2023-10-06T02:41:31.570 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-06T02:41:31.570 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-06T02:41:31.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:41:31.766 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default/ 2023-10-06T02:41:31.767 INFO:teuthology.task.install.deb:Package version is 17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:41:31.786 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default/ 2023-10-06T02:41:31.786 INFO:teuthology.task.install.deb:Package version is 17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:41:31.885 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:41:31.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-06T02:41:31.896 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:41:31.896 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-06T02:41:31.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2023-10-06T02:41:31.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2023-10-06T02:41:32.046 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T02:41:32.046 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T02:41:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T02:41:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T02:41:32.159 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal InRelease 2023-10-06T02:41:32.165 INFO:teuthology.orchestra.run.smithi167.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal InRelease 2023-10-06T02:41:32.181 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T02:41:32.181 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T02:41:32.206 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-06T02:41:32.207 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-06T02:41:32.246 INFO:teuthology.orchestra.run.smithi167.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal Release.gpg 2023-10-06T02:41:32.247 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal Release.gpg 2023-10-06T02:41:32.251 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T02:41:32.295 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main i386 Packages [3,380 B] 2023-10-06T02:41:32.295 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main i386 Packages [3,380 B] 2023-10-06T02:41:32.333 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T02:41:32.359 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-06T02:41:32.369 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-06T02:41:32.699 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 28.4 kB in 1s (44.6 kB/s) 2023-10-06T02:41:32.835 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 28.4 kB in 1s (37.0 kB/s) 2023-10-06T02:41:33.762 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:41:33.789 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1296-g736ac5ba-1focal cephadm=17.2.6-1296-g736ac5ba-1focal ceph-mds=17.2.6-1296-g736ac5ba-1focal ceph-mgr=17.2.6-1296-g736ac5ba-1focal ceph-common=17.2.6-1296-g736ac5ba-1focal ceph-fuse=17.2.6-1296-g736ac5ba-1focal ceph-test=17.2.6-1296-g736ac5ba-1focal ceph-volume=17.2.6-1296-g736ac5ba-1focal radosgw=17.2.6-1296-g736ac5ba-1focal python3-rados=17.2.6-1296-g736ac5ba-1focal python3-rgw=17.2.6-1296-g736ac5ba-1focal python3-cephfs=17.2.6-1296-g736ac5ba-1focal python3-rbd=17.2.6-1296-g736ac5ba-1focal libcephfs2=17.2.6-1296-g736ac5ba-1focal libcephfs-dev=17.2.6-1296-g736ac5ba-1focal librados2=17.2.6-1296-g736ac5ba-1focal librbd1=17.2.6-1296-g736ac5ba-1focal rbd-fuse=17.2.6-1296-g736ac5ba-1focal python3-cephfs=17.2.6-1296-g736ac5ba-1focal cephfs-shell=17.2.6-1296-g736ac5ba-1focal cephfs-top=17.2.6-1296-g736ac5ba-1focal cephfs-mirror=17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:41:33.859 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:41:33.875 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:41:33.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1296-g736ac5ba-1focal cephadm=17.2.6-1296-g736ac5ba-1focal ceph-mds=17.2.6-1296-g736ac5ba-1focal ceph-mgr=17.2.6-1296-g736ac5ba-1focal ceph-common=17.2.6-1296-g736ac5ba-1focal ceph-fuse=17.2.6-1296-g736ac5ba-1focal ceph-test=17.2.6-1296-g736ac5ba-1focal ceph-volume=17.2.6-1296-g736ac5ba-1focal radosgw=17.2.6-1296-g736ac5ba-1focal python3-rados=17.2.6-1296-g736ac5ba-1focal python3-rgw=17.2.6-1296-g736ac5ba-1focal python3-cephfs=17.2.6-1296-g736ac5ba-1focal python3-rbd=17.2.6-1296-g736ac5ba-1focal libcephfs2=17.2.6-1296-g736ac5ba-1focal libcephfs-dev=17.2.6-1296-g736ac5ba-1focal librados2=17.2.6-1296-g736ac5ba-1focal librbd1=17.2.6-1296-g736ac5ba-1focal rbd-fuse=17.2.6-1296-g736ac5ba-1focal python3-cephfs=17.2.6-1296-g736ac5ba-1focal cephfs-shell=17.2.6-1296-g736ac5ba-1focal cephfs-top=17.2.6-1296-g736ac5ba-1focal cephfs-mirror=17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:41:33.972 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:41:34.044 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2023-10-06T02:41:34.044 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2023-10-06T02:41:34.172 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2023-10-06T02:41:34.174 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2023-10-06T02:41:34.223 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-06T02:41:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-06T02:41:34.226 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T02:41:34.227 INFO:teuthology.orchestra.run.smithi033.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-06T02:41:34.228 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2023-10-06T02:41:34.229 INFO:teuthology.orchestra.run.smithi033.stdout: btrfs-tools 2023-10-06T02:41:34.348 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-06T02:41:34.349 INFO:teuthology.orchestra.run.smithi167.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-06T02:41:34.350 INFO:teuthology.orchestra.run.smithi167.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-06T02:41:34.351 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-06T02:41:34.351 INFO:teuthology.orchestra.run.smithi167.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-06T02:41:34.351 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T02:41:34.352 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-06T02:41:34.353 INFO:teuthology.orchestra.run.smithi167.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-06T02:41:34.354 INFO:teuthology.orchestra.run.smithi167.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-06T02:41:34.354 INFO:teuthology.orchestra.run.smithi167.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-06T02:41:34.354 INFO:teuthology.orchestra.run.smithi167.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-06T02:41:34.354 INFO:teuthology.orchestra.run.smithi167.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-06T02:41:34.354 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2023-10-06T02:41:34.354 INFO:teuthology.orchestra.run.smithi167.stdout: btrfs-tools 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T02:41:34.356 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T02:41:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T02:41:34.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T02:41:34.358 INFO:teuthology.orchestra.run.smithi033.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-06T02:41:34.358 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2023-10-06T02:41:34.358 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 librbd1 2023-10-06T02:41:34.424 INFO:teuthology.orchestra.run.smithi033.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-06T02:41:34.424 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 210 MB of archives. 2023-10-06T02:41:34.424 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-06T02:41:34.425 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-06T02:41:34.446 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-06T02:41:34.454 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-06T02:41:34.473 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2023-10-06T02:41:34.473 INFO:teuthology.orchestra.run.smithi167.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-06T02:41:34.473 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-06T02:41:34.474 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-06T02:41:34.474 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-06T02:41:34.474 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-06T02:41:34.474 INFO:teuthology.orchestra.run.smithi167.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-06T02:41:34.474 INFO:teuthology.orchestra.run.smithi167.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-06T02:41:34.475 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T02:41:34.476 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2023-10-06T02:41:34.477 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 librbd1 2023-10-06T02:41:34.484 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-06T02:41:34.507 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1296-g736ac5ba-1focal [3,501 kB] 2023-10-06T02:41:34.517 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-06T02:41:34.520 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-06T02:41:34.537 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-06T02:41:34.539 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-06T02:41:34.545 INFO:teuthology.orchestra.run.smithi167.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-06T02:41:34.545 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 210 MB of archives. 2023-10-06T02:41:34.545 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-06T02:41:34.545 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-06T02:41:34.562 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-06T02:41:34.563 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-06T02:41:34.566 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-06T02:41:34.569 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-06T02:41:34.574 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-06T02:41:34.614 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-06T02:41:34.615 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-06T02:41:34.618 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-06T02:41:34.624 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-06T02:41:34.626 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-06T02:41:34.626 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-06T02:41:34.629 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-06T02:41:34.629 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-06T02:41:34.630 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-06T02:41:34.631 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-06T02:41:34.636 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-06T02:41:34.652 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-06T02:41:34.652 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-06T02:41:34.652 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-06T02:41:34.653 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-06T02:41:34.654 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-06T02:41:34.655 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-06T02:41:34.655 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-06T02:41:34.655 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-06T02:41:34.657 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-06T02:41:34.658 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-06T02:41:34.659 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-06T02:41:34.669 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-06T02:41:34.672 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-06T02:41:34.675 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-06T02:41:34.676 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-06T02:41:34.681 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-06T02:41:34.681 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-06T02:41:34.696 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-06T02:41:34.697 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-06T02:41:34.698 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-06T02:41:34.699 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-06T02:41:34.701 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-06T02:41:34.702 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-06T02:41:34.703 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-06T02:41:34.704 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-06T02:41:34.712 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-06T02:41:34.715 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1296-g736ac5ba-1focal [3,554 kB] 2023-10-06T02:41:34.716 INFO:teuthology.orchestra.run.smithi033.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-06T02:41:34.717 INFO:teuthology.orchestra.run.smithi033.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-06T02:41:34.719 INFO:teuthology.orchestra.run.smithi033.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-06T02:41:34.719 INFO:teuthology.orchestra.run.smithi033.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-06T02:41:34.729 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1296-g736ac5ba-1focal [3,501 kB] 2023-10-06T02:41:34.735 INFO:teuthology.orchestra.run.smithi033.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-06T02:41:34.776 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-06T02:41:34.779 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-06T02:41:34.787 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-06T02:41:34.790 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-06T02:41:34.791 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-06T02:41:34.794 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-06T02:41:34.794 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-06T02:41:34.795 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-06T02:41:34.798 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-06T02:41:34.833 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-06T02:41:34.841 INFO:teuthology.orchestra.run.smithi033.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1296-g736ac5ba-1focal [760 kB] 2023-10-06T02:41:34.843 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-06T02:41:34.844 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-06T02:41:34.845 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-06T02:41:34.847 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-06T02:41:34.848 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-06T02:41:34.848 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-06T02:41:34.850 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-06T02:41:34.852 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-06T02:41:34.866 INFO:teuthology.orchestra.run.smithi033.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1296-g736ac5ba-1focal [413 kB] 2023-10-06T02:41:34.870 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-06T02:41:34.878 INFO:teuthology.orchestra.run.smithi033.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1296-g736ac5ba-1focal [22.5 kB] 2023-10-06T02:41:34.879 INFO:teuthology.orchestra.run.smithi033.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1296-g736ac5ba-1focal [200 kB] 2023-10-06T02:41:34.886 INFO:teuthology.orchestra.run.smithi033.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1296-g736ac5ba-1focal [47.1 kB] 2023-10-06T02:41:34.887 INFO:teuthology.orchestra.run.smithi033.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1296-g736ac5ba-1focal [386 kB] 2023-10-06T02:41:34.888 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-06T02:41:34.889 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-06T02:41:34.898 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-06T02:41:34.899 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-06T02:41:34.900 INFO:teuthology.orchestra.run.smithi033.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1296-g736ac5ba-1focal [5,570 kB] 2023-10-06T02:41:34.900 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-06T02:41:34.901 INFO:teuthology.orchestra.run.smithi033.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-06T02:41:34.901 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-06T02:41:34.902 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-06T02:41:34.904 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-06T02:41:34.911 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-06T02:41:34.936 INFO:teuthology.orchestra.run.smithi033.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-06T02:41:34.939 INFO:teuthology.orchestra.run.smithi033.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-06T02:41:34.952 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-06T02:41:34.955 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-06T02:41:34.958 INFO:teuthology.orchestra.run.smithi167.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-06T02:41:34.960 INFO:teuthology.orchestra.run.smithi167.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-06T02:41:34.961 INFO:teuthology.orchestra.run.smithi167.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-06T02:41:34.962 INFO:teuthology.orchestra.run.smithi167.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-06T02:41:34.965 INFO:teuthology.orchestra.run.smithi033.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-06T02:41:34.965 INFO:teuthology.orchestra.run.smithi033.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-06T02:41:34.966 INFO:teuthology.orchestra.run.smithi033.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-06T02:41:34.967 INFO:teuthology.orchestra.run.smithi033.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-06T02:41:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-06T02:41:34.984 INFO:teuthology.orchestra.run.smithi033.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-06T02:41:34.985 INFO:teuthology.orchestra.run.smithi033.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-06T02:41:34.988 INFO:teuthology.orchestra.run.smithi033.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-06T02:41:34.988 INFO:teuthology.orchestra.run.smithi033.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-06T02:41:35.021 INFO:teuthology.orchestra.run.smithi033.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-06T02:41:35.035 INFO:teuthology.orchestra.run.smithi033.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-06T02:41:35.037 INFO:teuthology.orchestra.run.smithi033.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-06T02:41:35.040 INFO:teuthology.orchestra.run.smithi033.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-06T02:41:35.041 INFO:teuthology.orchestra.run.smithi033.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-06T02:41:35.044 INFO:teuthology.orchestra.run.smithi033.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-06T02:41:35.046 INFO:teuthology.orchestra.run.smithi033.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-06T02:41:35.046 INFO:teuthology.orchestra.run.smithi033.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-06T02:41:35.091 INFO:teuthology.orchestra.run.smithi033.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1296-g736ac5ba-1focal [116 kB] 2023-10-06T02:41:35.095 INFO:teuthology.orchestra.run.smithi033.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1296-g736ac5ba-1focal [479 kB] 2023-10-06T02:41:35.111 INFO:teuthology.orchestra.run.smithi033.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1296-g736ac5ba-1focal [27.8 MB] 2023-10-06T02:41:35.137 INFO:teuthology.orchestra.run.smithi033.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-06T02:41:35.140 INFO:teuthology.orchestra.run.smithi033.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-06T02:41:35.185 INFO:teuthology.orchestra.run.smithi167.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-06T02:41:35.270 INFO:teuthology.orchestra.run.smithi033.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-06T02:41:35.271 INFO:teuthology.orchestra.run.smithi033.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-06T02:41:35.271 INFO:teuthology.orchestra.run.smithi033.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-06T02:41:35.272 INFO:teuthology.orchestra.run.smithi033.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-06T02:41:35.272 INFO:teuthology.orchestra.run.smithi033.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-06T02:41:35.273 INFO:teuthology.orchestra.run.smithi167.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-06T02:41:35.274 INFO:teuthology.orchestra.run.smithi033.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-06T02:41:35.276 INFO:teuthology.orchestra.run.smithi167.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-06T02:41:35.287 INFO:teuthology.orchestra.run.smithi033.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-06T02:41:35.287 INFO:teuthology.orchestra.run.smithi033.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-06T02:41:35.288 INFO:teuthology.orchestra.run.smithi167.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-06T02:41:35.288 INFO:teuthology.orchestra.run.smithi167.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-06T02:41:35.288 INFO:teuthology.orchestra.run.smithi167.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-06T02:41:35.289 INFO:teuthology.orchestra.run.smithi167.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-06T02:41:35.289 INFO:teuthology.orchestra.run.smithi167.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-06T02:41:35.290 INFO:teuthology.orchestra.run.smithi167.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-06T02:41:35.290 INFO:teuthology.orchestra.run.smithi167.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-06T02:41:35.293 INFO:teuthology.orchestra.run.smithi167.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-06T02:41:35.313 INFO:teuthology.orchestra.run.smithi033.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-06T02:41:35.313 INFO:teuthology.orchestra.run.smithi033.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-06T02:41:35.314 INFO:teuthology.orchestra.run.smithi167.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-06T02:41:35.315 INFO:teuthology.orchestra.run.smithi167.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-06T02:41:35.315 INFO:teuthology.orchestra.run.smithi033.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-06T02:41:35.316 INFO:teuthology.orchestra.run.smithi033.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-06T02:41:35.317 INFO:teuthology.orchestra.run.smithi033.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-06T02:41:35.319 INFO:teuthology.orchestra.run.smithi033.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-06T02:41:35.320 INFO:teuthology.orchestra.run.smithi033.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-06T02:41:35.320 INFO:teuthology.orchestra.run.smithi167.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-06T02:41:35.326 INFO:teuthology.orchestra.run.smithi167.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-06T02:41:35.326 INFO:teuthology.orchestra.run.smithi167.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-06T02:41:35.326 INFO:teuthology.orchestra.run.smithi167.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-06T02:41:35.332 INFO:teuthology.orchestra.run.smithi167.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-06T02:41:35.336 INFO:teuthology.orchestra.run.smithi167.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-06T02:41:35.337 INFO:teuthology.orchestra.run.smithi167.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-06T02:41:35.362 INFO:teuthology.orchestra.run.smithi167.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-06T02:41:35.365 INFO:teuthology.orchestra.run.smithi167.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-06T02:41:35.424 INFO:teuthology.orchestra.run.smithi167.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-06T02:41:35.425 INFO:teuthology.orchestra.run.smithi167.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-06T02:41:35.425 INFO:teuthology.orchestra.run.smithi167.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-06T02:41:35.425 INFO:teuthology.orchestra.run.smithi167.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-06T02:41:35.426 INFO:teuthology.orchestra.run.smithi167.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-06T02:41:35.427 INFO:teuthology.orchestra.run.smithi167.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-06T02:41:35.441 INFO:teuthology.orchestra.run.smithi167.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-06T02:41:35.442 INFO:teuthology.orchestra.run.smithi167.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-06T02:41:35.442 INFO:teuthology.orchestra.run.smithi167.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-06T02:41:35.448 INFO:teuthology.orchestra.run.smithi167.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-06T02:41:35.469 INFO:teuthology.orchestra.run.smithi167.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-06T02:41:35.471 INFO:teuthology.orchestra.run.smithi167.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-06T02:41:35.471 INFO:teuthology.orchestra.run.smithi167.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-06T02:41:35.474 INFO:teuthology.orchestra.run.smithi167.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-06T02:41:35.476 INFO:teuthology.orchestra.run.smithi167.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-06T02:41:36.150 INFO:teuthology.orchestra.run.smithi033.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1296-g736ac5ba-1focal [6,453 kB] 2023-10-06T02:41:36.276 INFO:teuthology.orchestra.run.smithi033.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1296-g736ac5ba-1focal [221 kB] 2023-10-06T02:41:36.283 INFO:teuthology.orchestra.run.smithi033.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1296-g736ac5ba-1focal [152 kB] 2023-10-06T02:41:36.290 INFO:teuthology.orchestra.run.smithi033.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1296-g736ac5ba-1focal [1,591 kB] 2023-10-06T02:41:36.344 INFO:teuthology.orchestra.run.smithi033.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1296-g736ac5ba-1focal [7,846 kB] 2023-10-06T02:41:36.617 INFO:teuthology.orchestra.run.smithi033.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1296-g736ac5ba-1focal [28.7 MB] 2023-10-06T02:41:37.583 INFO:teuthology.orchestra.run.smithi167.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1296-g736ac5ba-1focal [3,554 kB] 2023-10-06T02:41:37.755 INFO:teuthology.orchestra.run.smithi033.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1296-g736ac5ba-1focal [3,920 B] 2023-10-06T02:41:37.756 INFO:teuthology.orchestra.run.smithi033.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1296-g736ac5ba-1focal [869 kB] 2023-10-06T02:41:37.770 INFO:teuthology.orchestra.run.smithi033.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1296-g736ac5ba-1focal [2,408 kB] 2023-10-06T02:41:37.787 INFO:teuthology.orchestra.run.smithi033.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1296-g736ac5ba-1focal [79.1 kB] 2023-10-06T02:41:37.788 INFO:teuthology.orchestra.run.smithi033.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1296-g736ac5ba-1focal [117 kB] 2023-10-06T02:41:37.789 INFO:teuthology.orchestra.run.smithi033.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1296-g736ac5ba-1focal [1,760 kB] 2023-10-06T02:41:37.808 INFO:teuthology.orchestra.run.smithi033.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1296-g736ac5ba-1focal [7,888 kB] 2023-10-06T02:41:38.080 INFO:teuthology.orchestra.run.smithi033.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1296-g736ac5ba-1focal [4,032 B] 2023-10-06T02:41:38.080 INFO:teuthology.orchestra.run.smithi033.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1296-g736ac5ba-1focal [56.2 MB] 2023-10-06T02:41:40.074 INFO:teuthology.orchestra.run.smithi167.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1296-g736ac5ba-1focal [760 kB] 2023-10-06T02:41:40.305 INFO:teuthology.orchestra.run.smithi033.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1296-g736ac5ba-1focal [229 kB] 2023-10-06T02:41:40.310 INFO:teuthology.orchestra.run.smithi033.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1296-g736ac5ba-1focal [240 kB] 2023-10-06T02:41:40.315 INFO:teuthology.orchestra.run.smithi033.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1296-g736ac5ba-1focal [22.0 kB] 2023-10-06T02:41:40.316 INFO:teuthology.orchestra.run.smithi033.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1296-g736ac5ba-1focal [13.1 kB] 2023-10-06T02:41:40.317 INFO:teuthology.orchestra.run.smithi033.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1296-g736ac5ba-1focal [21.9 kB] 2023-10-06T02:41:40.317 INFO:teuthology.orchestra.run.smithi167.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1296-g736ac5ba-1focal [413 kB] 2023-10-06T02:41:40.317 INFO:teuthology.orchestra.run.smithi033.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1296-g736ac5ba-1focal [17.0 MB] 2023-10-06T02:41:40.533 INFO:teuthology.orchestra.run.smithi167.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1296-g736ac5ba-1focal [22.5 kB] 2023-10-06T02:41:40.541 INFO:teuthology.orchestra.run.smithi167.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1296-g736ac5ba-1focal [200 kB] 2023-10-06T02:41:40.619 INFO:teuthology.orchestra.run.smithi167.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1296-g736ac5ba-1focal [47.1 kB] 2023-10-06T02:41:40.665 INFO:teuthology.orchestra.run.smithi167.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1296-g736ac5ba-1focal [386 kB] 2023-10-06T02:41:40.838 INFO:teuthology.orchestra.run.smithi167.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1296-g736ac5ba-1focal [5,570 kB] 2023-10-06T02:41:40.878 INFO:teuthology.orchestra.run.smithi033.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1296-g736ac5ba-1focal [79.5 kB] 2023-10-06T02:41:41.521 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 210 MB in 7s (32.2 MB/s) 2023-10-06T02:41:41.669 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-06T02:41:41.728 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-06T02:41:41.732 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-06T02:41:41.775 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-06T02:41:42.070 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-06T02:41:42.090 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-06T02:41:42.128 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T02:41:42.248 INFO:teuthology.orchestra.run.smithi167.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1296-g736ac5ba-1focal [116 kB] 2023-10-06T02:41:42.280 INFO:teuthology.orchestra.run.smithi167.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1296-g736ac5ba-1focal [479 kB] 2023-10-06T02:41:42.396 INFO:teuthology.orchestra.run.smithi167.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1296-g736ac5ba-1focal [27.8 MB] 2023-10-06T02:41:42.423 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cmd2. 2023-10-06T02:41:42.442 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-06T02:41:42.482 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-06T02:41:42.826 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-06T02:41:42.843 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-06T02:41:42.926 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T02:41:43.299 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../004-librbd1_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:43.422 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librbd1 (17.2.6-1296-g736ac5ba-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T02:41:44.084 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../005-librados2_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:44.235 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librados2 (17.2.6-1296-g736ac5ba-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T02:41:44.889 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs2. 2023-10-06T02:41:44.907 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:44.947 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:45.258 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rados. 2023-10-06T02:41:45.275 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:45.316 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rados (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:45.670 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-06T02:41:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:41:45.728 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-argparse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cephfs. 2023-10-06T02:41:45.997 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:46.064 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cephfs (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:46.375 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-06T02:41:46.392 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:41:46.424 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:46.720 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-prettytable. 2023-10-06T02:41:46.736 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-06T02:41:46.778 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-06T02:41:46.970 INFO:teuthology.orchestra.run.smithi167.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1296-g736ac5ba-1focal [6,453 kB] 2023-10-06T02:41:47.047 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rbd. 2023-10-06T02:41:47.064 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:47.105 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rbd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:47.458 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-06T02:41:47.479 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-06T02:41:47.515 INFO:teuthology.orchestra.run.smithi167.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1296-g736ac5ba-1focal [221 kB] 2023-10-06T02:41:47.517 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T02:41:47.518 INFO:teuthology.orchestra.run.smithi167.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1296-g736ac5ba-1focal [152 kB] 2023-10-06T02:41:47.548 INFO:teuthology.orchestra.run.smithi167.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1296-g736ac5ba-1focal [1,591 kB] 2023-10-06T02:41:47.663 INFO:teuthology.orchestra.run.smithi167.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1296-g736ac5ba-1focal [7,846 kB] 2023-10-06T02:41:47.871 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-06T02:41:47.890 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-06T02:41:47.952 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T02:41:48.275 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-06T02:41:48.276 INFO:teuthology.orchestra.run.smithi167.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1296-g736ac5ba-1focal [28.7 MB] 2023-10-06T02:41:48.294 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-06T02:41:48.333 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T02:41:48.709 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-06T02:41:48.725 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-06T02:41:48.767 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T02:41:49.095 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-06T02:41:49.112 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-06T02:41:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T02:41:49.583 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-06T02:41:49.602 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T02:41:49.651 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:41:50.017 INFO:teuthology.orchestra.run.smithi167.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1296-g736ac5ba-1focal [3,920 B] 2023-10-06T02:41:50.018 INFO:teuthology.orchestra.run.smithi167.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1296-g736ac5ba-1focal [869 kB] 2023-10-06T02:41:50.027 INFO:teuthology.orchestra.run.smithi167.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1296-g736ac5ba-1focal [2,408 kB] 2023-10-06T02:41:50.091 INFO:teuthology.orchestra.run.smithi167.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1296-g736ac5ba-1focal [79.1 kB] 2023-10-06T02:41:50.092 INFO:teuthology.orchestra.run.smithi167.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1296-g736ac5ba-1focal [117 kB] 2023-10-06T02:41:50.095 INFO:teuthology.orchestra.run.smithi167.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1296-g736ac5ba-1focal [1,760 kB] 2023-10-06T02:41:50.166 INFO:teuthology.orchestra.run.smithi167.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1296-g736ac5ba-1focal [7,888 kB] 2023-10-06T02:41:50.170 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-06T02:41:50.190 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T02:41:50.227 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:41:50.497 INFO:teuthology.orchestra.run.smithi167.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1296-g736ac5ba-1focal [4,032 B] 2023-10-06T02:41:50.498 INFO:teuthology.orchestra.run.smithi167.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1296-g736ac5ba-1focal [56.2 MB] 2023-10-06T02:41:50.514 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-06T02:41:50.532 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T02:41:50.564 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:41:50.959 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-06T02:41:50.978 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-06T02:41:51.018 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T02:41:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librgw2. 2023-10-06T02:41:51.374 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../022-librgw2_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:51.412 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librgw2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:52.407 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rgw. 2023-10-06T02:41:52.426 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:52.465 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:52.730 INFO:teuthology.orchestra.run.smithi167.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1296-g736ac5ba-1focal [229 kB] 2023-10-06T02:41:52.735 INFO:teuthology.orchestra.run.smithi167.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1296-g736ac5ba-1focal [240 kB] 2023-10-06T02:41:52.739 INFO:teuthology.orchestra.run.smithi167.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1296-g736ac5ba-1focal [22.0 kB] 2023-10-06T02:41:52.741 INFO:teuthology.orchestra.run.smithi167.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1296-g736ac5ba-1focal [13.1 kB] 2023-10-06T02:41:52.742 INFO:teuthology.orchestra.run.smithi167.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1296-g736ac5ba-1focal [21.9 kB] 2023-10-06T02:41:52.743 INFO:teuthology.orchestra.run.smithi167.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1296-g736ac5ba-1focal [17.0 MB] 2023-10-06T02:41:52.834 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liboath0. 2023-10-06T02:41:52.852 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-06T02:41:52.893 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-06T02:41:53.162 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libradosstriper1. 2023-10-06T02:41:53.179 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:53.212 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libradosstriper1 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:53.284 INFO:teuthology.orchestra.run.smithi167.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1296-g736ac5ba-1focal [79.5 kB] 2023-10-06T02:41:53.657 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-common. 2023-10-06T02:41:53.673 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:53.749 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:53.894 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 210 MB in 19s (11.2 MB/s) 2023-10-06T02:41:54.064 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-06T02:41:54.118 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-06T02:41:54.122 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-06T02:41:54.162 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-06T02:41:54.490 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-06T02:41:54.507 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-06T02:41:54.548 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T02:41:54.843 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cmd2. 2023-10-06T02:41:54.862 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-06T02:41:54.893 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-06T02:41:55.262 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-06T02:41:55.279 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-06T02:41:55.363 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T02:41:55.757 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../004-librbd1_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:55.908 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librbd1 (17.2.6-1296-g736ac5ba-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T02:41:56.397 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-base. 2023-10-06T02:41:56.414 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:56.466 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-base (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:56.582 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../005-librados2_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:56.704 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librados2 (17.2.6-1296-g736ac5ba-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T02:41:57.401 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libcephfs2. 2023-10-06T02:41:57.419 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:57.459 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libcephfs2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:57.469 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cryptography. 2023-10-06T02:41:57.485 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-06T02:41:57.527 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T02:41:57.778 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rados. 2023-10-06T02:41:57.793 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:57.828 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rados (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:58.048 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-openssl. 2023-10-06T02:41:58.066 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-06T02:41:58.106 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-06T02:41:58.197 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-06T02:41:58.213 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:41:58.281 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-ceph-argparse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:58.367 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-06T02:41:58.383 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:41:58.425 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:58.542 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cephfs. 2023-10-06T02:41:58.557 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:58.625 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cephfs (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:58.822 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-06T02:41:58.838 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-06T02:41:58.880 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T02:41:58.996 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-06T02:41:59.014 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:41:59.046 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:59.174 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-06T02:41:59.194 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-06T02:41:59.233 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-06T02:41:59.392 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-prettytable. 2023-10-06T02:41:59.411 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-06T02:41:59.450 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-06T02:41:59.603 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-natsort. 2023-10-06T02:41:59.620 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-06T02:41:59.661 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-06T02:41:59.720 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rbd. 2023-10-06T02:41:59.738 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:41:59.780 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rbd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:41:59.990 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jwt. 2023-10-06T02:42:00.008 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-06T02:42:00.048 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T02:42:00.170 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-06T02:42:00.187 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-06T02:42:00.228 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T02:42:00.335 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-logutils. 2023-10-06T02:42:00.352 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-06T02:42:00.393 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-06T02:42:00.573 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-06T02:42:00.592 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-06T02:42:00.631 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T02:42:00.706 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-06T02:42:00.726 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-06T02:42:00.764 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-06T02:42:00.985 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-06T02:42:01.005 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-06T02:42:01.050 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-mako. 2023-10-06T02:42:01.065 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T02:42:01.070 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-06T02:42:01.109 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T02:42:01.454 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-06T02:42:01.471 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-06T02:42:01.474 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-06T02:42:01.488 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-06T02:42:01.512 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-06T02:42:01.531 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T02:42:01.815 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-06T02:42:01.832 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-06T02:42:01.849 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-06T02:42:01.868 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-06T02:42:01.873 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T02:42:01.899 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T02:42:02.159 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webob. 2023-10-06T02:42:02.178 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-06T02:42:02.217 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-06T02:42:02.370 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-06T02:42:02.388 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T02:42:02.438 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:42:02.587 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-06T02:42:02.604 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-06T02:42:02.645 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T02:42:02.932 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-06T02:42:02.948 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T02:42:02.948 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bs4. 2023-10-06T02:42:02.965 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-06T02:42:02.990 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:42:03.006 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-06T02:42:03.376 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-waitress. 2023-10-06T02:42:03.392 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-06T02:42:03.393 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-06T02:42:03.409 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T02:42:03.437 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T02:42:03.452 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:42:03.754 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-tempita. 2023-10-06T02:42:03.770 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-06T02:42:03.812 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-06T02:42:03.838 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-06T02:42:03.858 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-06T02:42:03.921 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T02:42:04.098 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-paste. 2023-10-06T02:42:04.118 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-06T02:42:04.156 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T02:42:04.266 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librgw2. 2023-10-06T02:42:04.284 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../022-librgw2_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:04.324 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librgw2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:04.476 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-06T02:42:04.494 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-06T02:42:04.534 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T02:42:04.837 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-06T02:42:04.853 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-06T02:42:04.895 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-06T02:42:05.119 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rgw. 2023-10-06T02:42:05.136 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:05.177 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:05.207 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webtest. 2023-10-06T02:42:05.224 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-06T02:42:05.281 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-06T02:42:05.564 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liboath0. 2023-10-06T02:42:05.580 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-06T02:42:05.592 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pecan. 2023-10-06T02:42:05.609 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-06T02:42:05.646 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-06T02:42:05.650 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T02:42:05.951 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libradosstriper1. 2023-10-06T02:42:05.968 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:05.989 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-06T02:42:06.005 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-06T02:42:06.034 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libradosstriper1 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T02:42:06.458 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-06T02:42:06.475 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:06.496 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-common. 2023-10-06T02:42:06.515 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:06.516 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:06.554 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:06.861 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr. 2023-10-06T02:42:06.877 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:06.911 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:07.456 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mon. 2023-10-06T02:42:07.473 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:07.515 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mon (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-osd. 2023-10-06T02:42:08.870 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:08.909 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-osd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:09.535 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-base. 2023-10-06T02:42:09.551 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:09.604 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-base (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:10.665 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cryptography. 2023-10-06T02:42:10.683 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-06T02:42:10.722 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T02:42:11.110 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-openssl. 2023-10-06T02:42:11.127 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-06T02:42:11.193 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-06T02:42:11.463 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-06T02:42:11.478 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:11.554 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:11.703 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph. 2023-10-06T02:42:11.720 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../055-ceph_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:11.753 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:11.977 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-06T02:42:11.993 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-06T02:42:12.035 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T02:42:12.114 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-fuse. 2023-10-06T02:42:12.131 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:12.189 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:12.371 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-06T02:42:12.388 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-06T02:42:12.429 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-06T02:42:12.617 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mds. 2023-10-06T02:42:12.634 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:12.675 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mds (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:12.800 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-natsort. 2023-10-06T02:42:12.816 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-06T02:42:12.858 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-06T02:42:13.195 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jwt. 2023-10-06T02:42:13.195 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephadm. 2023-10-06T02:42:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-06T02:42:13.212 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../058-cephadm_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:13.246 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:13.252 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T02:42:13.573 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jinja2. 2023-10-06T02:42:13.590 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-logutils. 2023-10-06T02:42:13.591 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-06T02:42:13.605 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-06T02:42:13.631 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-06T02:42:13.673 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-06T02:42:13.950 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-06T02:42:13.970 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:14.011 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-06T02:42:14.030 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-06T02:42:14.033 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:14.069 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-06T02:42:14.378 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-06T02:42:14.396 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-06T02:42:14.397 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-mako. 2023-10-06T02:42:14.413 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-06T02:42:14.436 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-06T02:42:14.455 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T02:42:14.672 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-underscore. 2023-10-06T02:42:14.690 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-06T02:42:14.730 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T02:42:14.809 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-06T02:42:14.825 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-06T02:42:14.867 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-06T02:42:15.024 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-06T02:42:15.041 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-06T02:42:15.082 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T02:42:15.212 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-06T02:42:15.228 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-06T02:42:15.270 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T02:42:15.402 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-routes. 2023-10-06T02:42:15.418 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-06T02:42:15.460 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-06T02:42:15.606 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webob. 2023-10-06T02:42:15.623 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-06T02:42:15.664 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-06T02:42:15.754 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-06T02:42:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:15.813 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:16.026 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-06T02:42:16.044 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-06T02:42:16.110 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T02:42:16.425 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-decorator. 2023-10-06T02:42:16.429 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-bs4. 2023-10-06T02:42:16.441 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-06T02:42:16.445 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-06T02:42:16.482 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T02:42:16.512 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-06T02:42:16.785 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-06T02:42:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-06T02:42:16.843 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T02:42:16.882 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-waitress. 2023-10-06T02:42:16.899 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-06T02:42:16.943 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T02:42:17.188 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-scipy. 2023-10-06T02:42:17.204 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-06T02:42:17.246 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-06T02:42:17.302 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-tempita. 2023-10-06T02:42:17.319 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-06T02:42:17.360 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-06T02:42:17.688 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-paste. 2023-10-06T02:42:17.704 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-06T02:42:17.746 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T02:42:18.093 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-06T02:42:18.110 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-06T02:42:18.150 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T02:42:18.495 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-06T02:42:18.512 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-06T02:42:18.530 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-06T02:42:18.550 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-06T02:42:18.578 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-06T02:42:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T02:42:18.881 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webtest. 2023-10-06T02:42:18.897 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-06T02:42:18.956 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-06T02:42:19.158 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-joblib. 2023-10-06T02:42:19.174 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-06T02:42:19.216 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-06T02:42:19.300 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pecan. 2023-10-06T02:42:19.317 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-06T02:42:19.358 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T02:42:19.594 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn. 2023-10-06T02:42:19.611 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-06T02:42:19.644 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T02:42:19.739 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-06T02:42:19.756 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-06T02:42:19.797 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T02:42:20.150 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-06T02:42:20.167 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:20.208 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:20.369 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-06T02:42:20.388 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:20.427 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:20.612 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr. 2023-10-06T02:42:20.628 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:20.695 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:21.207 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mon. 2023-10-06T02:42:21.224 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:21.265 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mon (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:21.538 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cachetools. 2023-10-06T02:42:21.554 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-06T02:42:21.596 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-06T02:42:21.924 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-06T02:42:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-06T02:42:21.982 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T02:42:22.310 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-06T02:42:22.328 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-06T02:42:22.360 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-osd. 2023-10-06T02:42:22.368 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T02:42:22.376 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:22.443 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-osd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:22.730 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rsa. 2023-10-06T02:42:22.750 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-06T02:42:22.788 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-06T02:42:23.083 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-google-auth. 2023-10-06T02:42:23.103 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-06T02:42:23.141 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-06T02:42:23.427 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-blinker. 2023-10-06T02:42:23.447 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-06T02:42:23.477 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T02:42:24.139 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-06T02:42:24.156 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-06T02:42:24.205 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T02:42:24.575 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-06T02:42:24.595 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-06T02:42:24.633 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T02:42:25.030 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-websocket. 2023-10-06T02:42:25.046 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-06T02:42:25.088 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T02:42:25.380 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph. 2023-10-06T02:42:25.398 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../055-ceph_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:25.400 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-06T02:42:25.416 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-06T02:42:25.458 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T02:42:25.472 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:25.825 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-fuse. 2023-10-06T02:42:25.843 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:25.882 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-06T02:42:26.072 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:26.113 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:26.320 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mds. 2023-10-06T02:42:26.336 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:26.369 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mds (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:26.425 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-06T02:42:26.441 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-06T02:42:26.516 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-06T02:42:26.911 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-06T02:42:26.914 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephadm. 2023-10-06T02:42:26.928 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T02:42:26.931 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../058-cephadm_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:26.969 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:42:26.972 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package jq. 2023-10-06T02:42:27.256 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T02:42:27.296 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:42:27.317 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jinja2. 2023-10-06T02:42:27.335 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-06T02:42:27.375 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-06T02:42:27.575 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package socat. 2023-10-06T02:42:27.592 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-06T02:42:27.632 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-06T02:42:27.720 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-06T02:42:27.736 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:27.778 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:27.944 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package xmlstarlet. 2023-10-06T02:42:27.962 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-06T02:42:28.002 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-06T02:42:28.148 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-06T02:42:28.164 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-06T02:42:28.206 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-06T02:42:28.348 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-test. 2023-10-06T02:42:28.365 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:28.431 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-test (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:28.484 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjs-underscore. 2023-10-06T02:42:28.503 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-06T02:42:28.542 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T02:42:28.862 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-06T02:42:28.879 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-06T02:42:28.964 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T02:42:29.317 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-routes. 2023-10-06T02:42:29.335 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-06T02:42:29.381 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-06T02:42:29.717 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-06T02:42:29.735 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:29.775 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:30.313 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-decorator. 2023-10-06T02:42:30.329 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-06T02:42:30.396 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T02:42:30.699 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-06T02:42:30.714 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-06T02:42:30.773 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T02:42:31.118 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-scipy. 2023-10-06T02:42:31.134 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-06T02:42:31.176 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-06T02:42:32.560 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-06T02:42:32.578 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-06T02:42:32.618 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T02:42:33.197 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-joblib. 2023-10-06T02:42:33.215 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-06T02:42:33.289 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-06T02:42:33.465 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-volume. 2023-10-06T02:42:33.484 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:33.523 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-volume (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:33.651 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-sklearn. 2023-10-06T02:42:33.670 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-06T02:42:33.709 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T02:42:33.929 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-06T02:42:33.949 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:33.987 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephfs-mirror (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:34.315 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-colorama. 2023-10-06T02:42:34.335 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-06T02:42:34.373 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-06T02:42:34.484 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-06T02:42:34.501 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:34.542 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:34.634 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephfs-shell. 2023-10-06T02:42:34.652 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:34.692 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephfs-shell (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:34.912 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephfs-top. 2023-10-06T02:42:34.929 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:34.961 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephfs-top (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:35.264 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-06T02:42:35.284 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T02:42:35.322 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:42:35.654 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cachetools. 2023-10-06T02:42:35.670 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-06T02:42:35.737 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-06T02:42:35.818 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-06T02:42:35.836 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:35.876 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs-dev (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:36.040 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-06T02:42:36.059 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-06T02:42:36.115 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T02:42:36.287 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package nvme-cli. 2023-10-06T02:42:36.307 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-06T02:42:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T02:42:36.451 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-06T02:42:36.468 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-06T02:42:36.510 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T02:42:36.729 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-06T02:42:36.746 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T02:42:36.787 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:42:36.863 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rsa. 2023-10-06T02:42:36.880 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-06T02:42:36.921 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-06T02:42:37.257 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-google-auth. 2023-10-06T02:42:37.275 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-06T02:42:37.315 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-06T02:42:37.660 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-blinker. 2023-10-06T02:42:37.678 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-06T02:42:37.718 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T02:42:38.038 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-06T02:42:38.058 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-06T02:42:38.096 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T02:42:38.475 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-06T02:42:38.491 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-06T02:42:38.492 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-06T02:42:38.510 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-06T02:42:38.549 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-06T02:42:38.558 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T02:42:38.877 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-06T02:42:38.894 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-06T02:42:38.943 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T02:42:38.989 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-websocket. 2023-10-06T02:42:39.005 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-06T02:42:39.047 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T02:42:39.254 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-attr. 2023-10-06T02:42:39.276 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-06T02:42:39.313 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-06T02:42:39.384 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-06T02:42:39.403 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-06T02:42:39.442 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T02:42:39.649 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webencodings. 2023-10-06T02:42:39.666 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-06T02:42:39.707 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T02:42:40.026 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-html5lib. 2023-10-06T02:42:40.045 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-06T02:42:40.082 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-06T02:42:40.084 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-06T02:42:40.098 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:40.140 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:40.388 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-06T02:42:40.407 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-06T02:42:40.437 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T02:42:40.493 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-06T02:42:40.511 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-06T02:42:40.551 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-06T02:42:40.924 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-packaging. 2023-10-06T02:42:40.938 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-06T02:42:40.944 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-06T02:42:40.954 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T02:42:40.982 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-06T02:42:41.021 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:42:41.293 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastescript. 2023-10-06T02:42:41.307 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package jq. 2023-10-06T02:42:41.310 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-06T02:42:41.324 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T02:42:41.351 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-06T02:42:41.390 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:42:41.643 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package socat. 2023-10-06T02:42:41.655 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pluggy. 2023-10-06T02:42:41.660 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-06T02:42:41.675 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-06T02:42:41.713 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-06T02:42:41.750 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-06T02:42:42.088 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package xmlstarlet. 2023-10-06T02:42:42.105 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-06T02:42:42.146 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-06T02:42:42.175 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-psutil. 2023-10-06T02:42:42.193 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-06T02:42:42.233 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T02:42:42.518 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-test. 2023-10-06T02:42:42.537 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:42.552 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-py. 2023-10-06T02:42:42.572 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-06T02:42:42.575 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-test (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:42.610 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T02:42:42.964 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-06T02:42:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-06T02:42:43.021 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T02:42:43.325 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pytest. 2023-10-06T02:42:43.342 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-06T02:42:43.383 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-06T02:42:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplejson. 2023-10-06T02:42:43.748 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-06T02:42:43.820 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T02:42:44.072 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-06T02:42:44.092 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-06T02:42:44.130 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T02:42:44.863 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package radosgw. 2023-10-06T02:42:44.884 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../114-radosgw_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:44.921 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking radosgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:46.724 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package rbd-fuse. 2023-10-06T02:42:46.743 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:46.782 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking rbd-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:47.080 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-06T02:42:47.387 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T02:42:47.513 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T02:42:47.555 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-volume. 2023-10-06T02:42:47.572 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:47.638 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-volume (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:47.664 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T02:42:47.858 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:48.069 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-06T02:42:48.087 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:48.127 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephfs-mirror (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user cephadm....done 2023-10-06T02:42:48.497 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-colorama. 2023-10-06T02:42:48.516 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-06T02:42:48.555 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-06T02:42:48.626 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:42:48.736 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T02:42:48.825 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephfs-shell. 2023-10-06T02:42:48.841 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:48.883 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephfs-shell (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:49.080 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-06T02:42:49.166 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephfs-top. 2023-10-06T02:42:49.183 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1296-g736ac5ba-1focal_all.deb ... 2023-10-06T02:42:49.224 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephfs-top (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:49.364 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:42:49.490 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-06T02:42:49.509 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-06T02:42:49.525 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T02:42:49.567 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:42:49.634 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T02:42:49.975 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-06T02:42:50.145 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-06T02:42:50.161 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:50.203 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libcephfs-dev (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:50.355 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T02:42:50.623 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package nvme-cli. 2023-10-06T02:42:50.639 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-06T02:42:50.640 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-06T02:42:50.699 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T02:42:50.941 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-06T02:42:51.045 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-06T02:42:51.063 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T02:42:51.136 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:42:51.242 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T02:42:51.509 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T02:42:51.794 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-argparse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:52.103 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T02:42:52.229 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-06T02:42:52.530 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-06T02:42:52.819 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-06T02:42:52.831 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T02:42:52.836 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-06T02:42:52.910 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-06T02:42:53.141 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T02:42:53.263 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-06T02:42:53.267 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T02:42:53.280 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-06T02:42:53.321 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T02:42:53.623 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-attr. 2023-10-06T02:42:53.626 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-06T02:42:53.641 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-06T02:42:53.681 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-06T02:42:53.936 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T02:42:54.035 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webencodings. 2023-10-06T02:42:54.052 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-06T02:42:54.093 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T02:42:54.237 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T02:42:54.429 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-html5lib. 2023-10-06T02:42:54.445 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-06T02:42:54.487 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-06T02:42:54.596 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T02:42:54.958 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-06T02:42:54.978 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-06T02:42:55.041 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T02:42:55.123 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-06T02:42:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-06T02:42:55.544 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-packaging. 2023-10-06T02:42:55.561 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-06T02:42:55.602 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-06T02:42:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-06T02:42:55.930 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pastescript. 2023-10-06T02:42:55.947 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-06T02:42:55.988 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-06T02:42:56.333 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T02:42:56.333 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T02:42:56.350 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pluggy. 2023-10-06T02:42:56.861 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T02:42:56.861 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T02:42:56.862 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-06T02:42:56.862 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-06T02:42:56.862 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-psutil. 2023-10-06T02:42:56.862 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-06T02:42:56.862 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T02:42:57.073 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-06T02:42:57.215 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-py. 2023-10-06T02:42:57.233 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-06T02:42:57.273 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T02:42:57.400 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T02:42:57.629 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-06T02:42:57.648 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-06T02:42:57.687 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T02:42:57.778 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T02:42:57.990 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pytest. 2023-10-06T02:42:58.008 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-06T02:42:58.048 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-06T02:42:58.146 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-06T02:42:58.280 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:42:58.410 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-simplejson. 2023-10-06T02:42:58.429 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-06T02:42:58.468 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T02:42:58.598 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:42:58.732 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-06T02:42:58.771 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-06T02:42:58.789 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-06T02:42:58.829 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T02:42:59.095 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T02:42:59.246 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-06T02:42:59.364 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-06T02:42:59.648 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T02:42:59.658 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package radosgw. 2023-10-06T02:42:59.676 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../114-radosgw_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:42:59.975 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:43:00.026 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking radosgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:00.134 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T02:43:00.260 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T02:43:00.588 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T02:43:00.856 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-06T02:43:01.165 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T02:43:01.291 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-06T02:43:01.592 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-06T02:43:01.804 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package rbd-fuse. 2023-10-06T02:43:01.821 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1296-g736ac5ba-1focal_amd64.deb ... 2023-10-06T02:43:01.862 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking rbd-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:01.910 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T02:43:02.198 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-06T02:43:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T02:43:02.488 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T02:43:02.614 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T02:43:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-06T02:43:02.740 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T02:43:02.872 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-06T02:43:02.975 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:03.015 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T02:43:03.333 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-06T02:43:03.639 INFO:teuthology.orchestra.run.smithi167.stdout:Adding system user cephadm....done 2023-10-06T02:43:03.692 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T02:43:03.767 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:43:03.818 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T02:43:03.918 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T02:43:03.978 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-06T02:43:04.245 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-06T02:43:04.304 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T02:43:04.512 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T02:43:04.605 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librados2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:04.672 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-06T02:43:04.731 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-06T02:43:04.798 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T02:43:05.149 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-06T02:43:05.577 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T02:43:05.886 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-06T02:43:06.196 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-06T02:43:06.253 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:06.362 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:43:06.472 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T02:43:06.488 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-06T02:43:06.765 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T02:43:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-06T02:43:07.074 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-ceph-argparse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:07.182 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-06T02:43:07.392 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T02:43:07.476 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:43:07.518 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-06T02:43:07.601 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-06T02:43:07.794 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-06T02:43:07.952 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:08.086 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T02:43:08.087 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T02:43:08.363 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T02:43:08.397 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libradosstriper1 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:08.514 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T02:43:08.523 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T02:43:08.832 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T02:43:08.891 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-06T02:43:09.184 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T02:43:09.483 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T02:43:09.789 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T02:43:09.897 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librbd1 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:10.022 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T02:43:10.098 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-06T02:43:10.140 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-06T02:43:10.241 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:43:10.367 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-06T02:43:10.475 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-06T02:43:10.701 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T02:43:10.792 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-06T02:43:10.836 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-06T02:43:11.152 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T02:43:11.231 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-modules-core (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:11.311 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T02:43:11.348 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:11.440 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T02:43:11.562 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T02:43:11.562 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T02:43:11.750 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T02:43:12.000 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs-dev (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:12.109 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T02:43:12.143 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-06T02:43:12.410 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-06T02:43:12.444 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T02:43:12.739 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T02:43:12.804 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T02:43:13.118 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rados (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:13.189 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-06T02:43:13.225 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librgw2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:13.317 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:13.352 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephfs-top (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:13.477 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rbd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:13.645 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up rbd-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:13.652 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:43:13.780 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-06T02:43:13.828 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-06T02:43:14.081 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-06T02:43:14.179 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T02:43:14.305 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-06T02:43:14.391 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:14.456 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-06T02:43:14.510 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cephfs (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:14.635 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephfs-shell (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:14.761 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T02:43:14.782 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T02:43:15.125 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:43:15.252 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T02:43:15.378 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T02:43:15.704 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T02:43:15.718 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:15.980 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-06T02:43:16.256 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T02:43:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:Adding group ceph....done 2023-10-06T02:43:16.382 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-06T02:43:16.639 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user ceph....done 2023-10-06T02:43:16.667 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-06T02:43:16.816 INFO:teuthology.orchestra.run.smithi033.stdout:Setting system user ceph properties....done 2023-10-06T02:43:16.840 INFO:teuthology.orchestra.run.smithi033.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-06T02:43:16.959 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T02:43:16.989 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-06T02:43:17.261 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T02:43:17.300 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-06T02:43:17.653 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-06T02:43:17.745 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephfs-mirror (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:17.947 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-06T02:43:17.978 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T02:43:17.978 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T02:43:18.073 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T02:43:18.390 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-06T02:43:18.430 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-test (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:18.556 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up radosgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:18.750 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T02:43:18.819 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T02:43:18.819 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T02:43:18.876 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T02:43:19.036 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-06T02:43:19.387 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T02:43:19.502 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-base (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:19.655 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librados2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:19.797 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-06T02:43:19.874 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-06T02:43:20.338 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mds (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:20.570 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T02:43:20.571 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T02:43:21.032 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:21.261 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:21.269 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T02:43:21.270 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T02:43:21.387 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:43:21.530 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-06T02:43:21.717 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-osd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:21.906 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-06T02:43:22.207 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-06T02:43:22.209 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T02:43:22.210 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T02:43:22.542 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T02:43:22.668 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-06T02:43:22.702 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:22.828 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:22.971 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mon (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:23.028 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libcephfs2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:23.162 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T02:43:23.206 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T02:43:23.206 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T02:43:23.506 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libradosstriper1 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:23.665 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T02:43:23.665 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:23.791 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:23.917 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-dashboard (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:23.999 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T02:43:24.060 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-volume (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:24.221 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T02:43:24.364 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T02:43:24.657 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T02:43:25.038 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librbd1 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:25.164 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T02:43:25.290 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-06T02:43:25.449 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T02:43:25.575 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-06T02:43:25.903 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T02:43:26.054 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-06T02:43:26.450 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-modules-core (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:26.576 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:26.801 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T02:43:26.801 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T02:43:27.260 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libcephfs-dev (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:27.386 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T02:43:27.618 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-06T02:43:27.738 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-06T02:43:28.081 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T02:43:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T02:43:28.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-06T02:43:28.301 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2023-10-06T02:43:28.500 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rados (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:28.510 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2023-10-06T02:43:28.511 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2023-10-06T02:43:28.626 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librgw2 (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:28.723 INFO:teuthology.orchestra.run.smithi033.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-06T02:43:28.723 INFO:teuthology.orchestra.run.smithi033.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-06T02:43:28.723 INFO:teuthology.orchestra.run.smithi033.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-06T02:43:28.723 INFO:teuthology.orchestra.run.smithi033.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-06T02:43:28.724 INFO:teuthology.orchestra.run.smithi033.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-06T02:43:28.724 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T02:43:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T02:43:28.724 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T02:43:28.725 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2023-10-06T02:43:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-06T02:43:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-06T02:43:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-06T02:43:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: usb-modeswitch-data wpasupplicant 2023-10-06T02:43:28.727 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2023-10-06T02:43:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-06T02:43:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-06T02:43:28.753 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephfs-top (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:28.779 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2023-10-06T02:43:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-06T02:43:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-06T02:43:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-06T02:43:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-06T02:43:28.781 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2023-10-06T02:43:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-06T02:43:28.857 INFO:teuthology.orchestra.run.smithi033.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-06T02:43:28.857 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 10.4 MB of archives. 2023-10-06T02:43:28.857 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-06T02:43:28.857 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-06T02:43:28.915 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rbd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-06T02:43:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-06T02:43:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-06T02:43:28.992 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-06T02:43:28.993 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-06T02:43:29.009 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-06T02:43:29.009 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-06T02:43:29.010 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-06T02:43:29.011 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-06T02:43:29.012 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-06T02:43:29.017 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-06T02:43:29.018 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-06T02:43:29.020 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-06T02:43:29.020 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-06T02:43:29.034 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-06T02:43:29.035 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-06T02:43:29.040 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-06T02:43:29.041 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up rbd-fuse (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:29.048 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-06T02:43:29.049 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-06T02:43:29.064 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-06T02:43:29.064 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-06T02:43:29.072 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-06T02:43:29.082 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-06T02:43:29.098 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-06T02:43:29.102 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-06T02:43:29.103 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-06T02:43:29.103 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-06T02:43:29.103 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-06T02:43:29.189 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-06T02:43:29.430 INFO:teuthology.orchestra.run.smithi033.stdout:Preconfiguring packages ... 2023-10-06T02:43:29.493 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-06T02:43:29.565 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 10.4 MB in 0s (33.5 MB/s) 2023-10-06T02:43:29.836 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:29.844 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-06T02:43:29.849 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T02:43:29.988 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cephfs (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:30.019 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T02:43:30.115 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephfs-shell (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:30.281 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T02:43:31.227 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-common (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:31.264 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-06T02:43:31.824 INFO:teuthology.orchestra.run.smithi167.stdout:Adding group ceph....done 2023-10-06T02:43:32.242 INFO:teuthology.orchestra.run.smithi167.stdout:Adding system user ceph....done 2023-10-06T02:43:32.452 INFO:teuthology.orchestra.run.smithi167.stdout:Setting system user ceph properties....done 2023-10-06T02:43:32.475 INFO:teuthology.orchestra.run.smithi167.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-06T02:43:32.626 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-06T02:43:32.750 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-06T02:43:32.755 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T02:43:32.869 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T02:43:32.944 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-06T02:43:33.272 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T02:43:33.381 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephfs-mirror (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T02:43:33.640 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T02:43:33.640 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T02:43:33.805 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T02:43:34.058 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-test (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:34.200 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up radosgw (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:34.243 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-06T02:43:34.243 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T02:43:34.392 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T02:43:34.472 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T02:43:34.472 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T02:43:34.866 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T02:43:35.006 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T02:43:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-base (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:35.420 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T02:43:35.497 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-06T02:43:35.553 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-06T02:43:36.032 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mds (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:36.040 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-06T02:43:36.058 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-06T02:43:36.099 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-06T02:43:36.258 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T02:43:36.259 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T02:43:36.532 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package dump. 2023-10-06T02:43:36.552 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-06T02:43:36.708 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:36.811 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking dump (0.4b46-6) ... 2023-10-06T02:43:36.946 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T02:43:36.947 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T02:43:37.115 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package indent. 2023-10-06T02:43:37.136 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T02:43:37.173 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T02:43:37.401 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-osd (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:37.485 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-06T02:43:37.503 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-06T02:43:37.543 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T02:43:37.887 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T02:43:37.887 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T02:43:37.913 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-06T02:43:37.931 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-06T02:43:37.972 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T02:43:38.233 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-06T02:43:38.252 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T02:43:38.291 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:43:38.320 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:38.479 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:38.622 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mon (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-06T02:43:38.688 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:38.727 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:43:38.883 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T02:43:38.883 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T02:43:38.997 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-06T02:43:39.014 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:39.055 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:43:39.323 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-cephadm (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:39.383 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-06T02:43:39.401 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:39.441 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:43:39.491 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:39.617 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-dashboard (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:39.776 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-volume (17.2.6-1296-g736ac5ba-1focal) ... 2023-10-06T02:43:39.828 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-06T02:43:39.845 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-06T02:43:39.886 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T02:43:39.887 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T02:43:40.030 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T02:43:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-06T02:43:40.217 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T02:43:40.256 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T02:43:40.290 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T02:43:40.659 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-06T02:43:40.677 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:40.717 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:43:41.045 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-06T02:43:41.063 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:41.103 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:43:41.339 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-06T02:43:41.356 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T02:43:41.389 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T02:43:42.068 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-06T02:43:42.088 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-06T02:43:42.200 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-06T02:43:42.723 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package modemmanager. 2023-10-06T02:43:42.740 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:42.791 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:43:43.234 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-06T02:43:43.329 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package wpasupplicant. 2023-10-06T02:43:43.347 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-06T02:43:43.378 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T02:43:43.851 INFO:teuthology.orchestra.run.smithi167.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T02:43:43.854 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-06T02:43:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package network-manager. 2023-10-06T02:43:43.921 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2023-10-06T02:43:43.934 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T02:43:44.000 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T02:43:44.125 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2023-10-06T02:43:44.125 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2023-10-06T02:43:44.315 INFO:teuthology.orchestra.run.smithi167.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-06T02:43:44.315 INFO:teuthology.orchestra.run.smithi167.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-06T02:43:44.315 INFO:teuthology.orchestra.run.smithi167.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-06T02:43:44.316 INFO:teuthology.orchestra.run.smithi167.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-06T02:43:44.316 INFO:teuthology.orchestra.run.smithi167.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-06T02:43:44.316 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T02:43:44.316 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T02:43:44.316 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T02:43:44.316 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2023-10-06T02:43:44.317 INFO:teuthology.orchestra.run.smithi167.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-06T02:43:44.317 INFO:teuthology.orchestra.run.smithi167.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-06T02:43:44.317 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-06T02:43:44.317 INFO:teuthology.orchestra.run.smithi167.stdout: usb-modeswitch-data wpasupplicant 2023-10-06T02:43:44.318 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2023-10-06T02:43:44.318 INFO:teuthology.orchestra.run.smithi167.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-06T02:43:44.318 INFO:teuthology.orchestra.run.smithi167.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-06T02:43:44.362 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2023-10-06T02:43:44.362 INFO:teuthology.orchestra.run.smithi167.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-06T02:43:44.363 INFO:teuthology.orchestra.run.smithi167.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-06T02:43:44.363 INFO:teuthology.orchestra.run.smithi167.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-06T02:43:44.363 INFO:teuthology.orchestra.run.smithi167.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-06T02:43:44.364 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2023-10-06T02:43:44.365 INFO:teuthology.orchestra.run.smithi167.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-06T02:43:44.423 INFO:teuthology.orchestra.run.smithi167.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-06T02:43:44.423 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 10.4 MB of archives. 2023-10-06T02:43:44.423 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-06T02:43:44.423 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-06T02:43:44.573 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-06T02:43:44.574 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-06T02:43:44.575 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-06T02:43:44.581 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-06T02:43:44.582 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-06T02:43:44.601 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-06T02:43:44.602 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-06T02:43:44.603 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-06T02:43:44.605 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-06T02:43:44.605 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-06T02:43:44.611 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-06T02:43:44.612 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-06T02:43:44.614 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-06T02:43:44.614 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-06T02:43:44.631 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-06T02:43:44.632 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-06T02:43:44.640 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-06T02:43:44.646 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-06T02:43:44.646 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-06T02:43:44.660 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-06T02:43:44.660 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-06T02:43:44.667 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-06T02:43:44.677 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-06T02:43:44.692 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-06T02:43:44.695 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-06T02:43:44.696 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-06T02:43:44.696 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-06T02:43:44.696 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-06T02:43:44.733 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ppp. 2023-10-06T02:43:44.751 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-06T02:43:45.005 INFO:teuthology.orchestra.run.smithi167.stdout:Preconfiguring packages ... 2023-10-06T02:43:45.027 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T02:43:45.155 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 10.4 MB in 0s (32.1 MB/s) 2023-10-06T02:43:45.413 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package pptp-linux. 2023-10-06T02:43:45.431 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-06T02:43:45.465 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-06T02:43:45.470 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T02:43:45.472 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-06T02:43:45.645 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T02:43:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-06T02:43:45.851 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-06T02:43:45.910 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-06T02:43:46.145 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-06T02:43:46.162 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-06T02:43:46.203 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-06T02:43:46.590 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-06T02:43:46.609 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-06T02:43:46.659 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T02:43:46.972 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-06T02:43:46.973 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T02:43:47.284 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-06T02:43:47.388 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T02:43:47.707 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T02:43:48.398 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-06T02:43:48.403 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T02:43:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T02:43:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T02:43:48.548 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T02:43:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T02:43:48.961 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T02:43:49.037 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-06T02:43:49.125 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T02:43:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T02:43:49.298 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-06T02:43:49.424 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T02:43:49.485 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T02:43:49.550 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T02:43:49.896 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-06T02:43:49.901 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T02:43:50.067 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T02:43:50.473 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-06T02:43:50.549 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T02:43:50.698 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up dump (0.4b46-6) ... 2023-10-06T02:43:50.714 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T02:43:50.793 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-06T02:43:50.858 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:43:50.984 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T02:43:51.121 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T02:43:51.245 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T02:43:51.400 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T02:43:51.421 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T02:43:51.547 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:43:51.654 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-06T02:43:51.673 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:43:51.799 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-06T02:43:51.968 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:43:52.119 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T02:43:52.208 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-06T02:43:52.225 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-06T02:43:52.266 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-06T02:43:52.594 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package dump. 2023-10-06T02:43:52.611 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-06T02:43:52.644 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking dump (0.4b46-6) ... 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout:which means they are currently configured by ifupdown: 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout:- ens1f0 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout:remove their configuration from /etc/network/interfaces. 2023-10-06T02:43:52.833 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:43:52.956 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package indent. 2023-10-06T02:43:52.973 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T02:43:52.994 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-06T02:43:53.014 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T02:43:53.340 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-06T02:43:53.367 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-06T02:43:53.385 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-06T02:43:53.425 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T02:43:53.762 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-06T02:43:53.782 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-06T02:43:53.821 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T02:43:53.867 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-06T02:43:54.090 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-06T02:43:54.108 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T02:43:54.148 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:43:54.364 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:43:54.507 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-06T02:43:54.518 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-06T02:43:54.535 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:54.601 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:43:54.650 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:43:54.776 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:43:54.888 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-06T02:43:54.893 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:43:54.905 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:54.962 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:43:55.045 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:43:55.270 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-06T02:43:55.291 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:55.314 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T02:43:55.314 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T02:43:55.332 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:43:55.735 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-06T02:43:55.753 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-06T02:43:55.793 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T02:43:55.848 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-06T02:43:55.965 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-06T02:43:56.091 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T02:43:56.163 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-06T02:43:56.181 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T02:43:56.221 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T02:43:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T02:43:56.536 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T02:43:56.802 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-06T02:43:56.820 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:56.859 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:43:57.228 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-06T02:43:57.246 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:57.286 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:43:57.572 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-06T02:43:57.590 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T02:43:57.631 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T02:43:58.168 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-06T02:43:58.185 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-06T02:43:58.226 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-06T02:43:58.629 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package modemmanager. 2023-10-06T02:43:58.646 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T02:43:58.698 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:43:59.402 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package wpasupplicant. 2023-10-06T02:43:59.419 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-06T02:43:59.460 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T02:44:00.016 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package network-manager. 2023-10-06T02:44:00.034 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T02:44:00.095 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T02:44:00.460 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-06T02:44:00.572 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-06T02:44:00.807 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ppp. 2023-10-06T02:44:00.823 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-06T02:44:01.095 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T02:44:01.453 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package pptp-linux. 2023-10-06T02:44:01.473 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-06T02:44:01.512 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-06T02:44:01.555 INFO:teuthology.orchestra.run.smithi033.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T02:44:01.558 DEBUG:teuthology.parallel:result is None 2023-10-06T02:44:02.221 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-06T02:44:02.240 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-06T02:44:02.518 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-06T02:44:03.028 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-06T02:44:03.046 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-06T02:44:03.086 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-06T02:44:03.523 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-06T02:44:03.543 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-06T02:44:03.592 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T02:44:03.928 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T02:44:04.279 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-06T02:44:04.405 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T02:44:04.557 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T02:44:04.954 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T02:44:04.954 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T02:44:05.409 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T02:44:05.535 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-06T02:44:05.678 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T02:44:05.796 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-06T02:44:05.922 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T02:44:06.081 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T02:44:07.080 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-06T02:44:07.321 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up dump (0.4b46-6) ... 2023-10-06T02:44:07.467 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-06T02:44:07.531 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:44:07.657 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T02:44:07.920 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T02:44:08.161 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T02:44:08.336 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T02:44:08.487 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:44:08.622 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:44:08.748 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-06T02:44:08.941 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:44:09.067 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T02:44:09.788 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:09.788 INFO:teuthology.orchestra.run.smithi167.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-06T02:44:09.788 INFO:teuthology.orchestra.run.smithi167.stdout:which means they are currently configured by ifupdown: 2023-10-06T02:44:09.788 INFO:teuthology.orchestra.run.smithi167.stdout:- enp3s0f1 2023-10-06T02:44:09.788 INFO:teuthology.orchestra.run.smithi167.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-06T02:44:09.788 INFO:teuthology.orchestra.run.smithi167.stdout:remove their configuration from /etc/network/interfaces. 2023-10-06T02:44:09.789 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:09.948 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-06T02:44:10.257 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-06T02:44:10.791 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-06T02:44:11.296 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T02:44:11.447 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-06T02:44:11.624 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T02:44:11.749 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:44:11.876 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T02:44:12.026 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T02:44:12.300 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T02:44:12.300 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T02:44:12.804 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-06T02:44:12.905 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-06T02:44:13.058 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T02:44:13.210 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T02:44:13.486 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T02:44:17.397 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-06T02:44:17.497 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-06T02:44:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T02:44:18.433 DEBUG:teuthology.parallel:result is None 2023-10-06T02:44:18.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:44:18.661 DEBUG:teuthology.orchestra.run.smithi033:> dpkg-query -W -f '${Version}' ceph 2023-10-06T02:44:18.686 INFO:teuthology.orchestra.run.smithi033.stdout:17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:44:18.687 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:44:18.687 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296-g736ac5ba-1focal is installed. 2023-10-06T02:44:18.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:44:18.834 DEBUG:teuthology.orchestra.run.smithi167:> dpkg-query -W -f '${Version}' ceph 2023-10-06T02:44:18.859 INFO:teuthology.orchestra.run.smithi167.stdout:17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:44:18.859 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296-g736ac5ba-1focal 2023-10-06T02:44:18.860 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296-g736ac5ba-1focal is installed. 2023-10-06T02:44:18.860 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T02:44:18.860 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:18.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T02:44:18.877 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:18.877 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T02:44:18.892 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T02:44:18.893 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:18.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T02:44:18.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T02:44:18.918 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:18.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T02:44:18.947 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T02:44:18.959 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T02:44:18.959 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:18.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T02:44:18.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T02:44:18.988 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:18.988 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T02:44:19.004 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T02:44:19.065 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T02:44:19.066 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:19.066 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T02:44:19.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T02:44:19.141 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:19.141 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T02:44:19.157 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T02:44:19.172 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T02:44:19.235 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T02:44:19.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2023-10-06T02:44:19.237 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2023-10-06T02:44:19.253 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T02:44:19.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T02:44:19.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T02:44:19.311 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T02:44:19.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T02:44:19.314 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T02:44:19.333 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T02:44:19.333 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T02:44:19.333 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T02:44:19.333 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:19.333 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T02:44:19.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T02:44:19.419 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:19.419 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T02:44:19.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T02:44:19.512 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T02:44:19.513 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T02:44:19.513 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156', 'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412156', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7519, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f', 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:52:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T02:44:19.513 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T02:44:19.516 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T02:44:19.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T02:44:19.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T02:44:19.581 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:19.581 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T02:44:19.587 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T02:44:19.588 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 360 Links: 1 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 02:39:54.511200986 +0000 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 02:39:54.243207167 +0000 2023-10-06T02:44:19.594 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 02:39:54.243207167 +0000 2023-10-06T02:44:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T02:44:19.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T02:44:19.608 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T02:44:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T02:44:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000411628 s, 1.2 MB/s 2023-10-06T02:44:19.609 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T02:44:19.618 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2023-10-06T02:44:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T02:44:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.669 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 364 Links: 1 2023-10-06T02:44:19.669 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.669 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 02:39:55.019189271 +0000 2023-10-06T02:44:19.669 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 02:39:54.747195544 +0000 2023-10-06T02:44:19.669 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 02:39:54.747195544 +0000 2023-10-06T02:44:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T02:44:19.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T02:44:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T02:44:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T02:44:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000376938 s, 1.4 MB/s 2023-10-06T02:44:19.681 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T02:44:19.688 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2023-10-06T02:44:19.736 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T02:44:19.736 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.736 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-10-06T02:44:19.737 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.737 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 02:39:55.231184383 +0000 2023-10-06T02:44:19.737 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 02:39:55.227184474 +0000 2023-10-06T02:44:19.737 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 02:39:55.227184474 +0000 2023-10-06T02:44:19.737 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T02:44:19.737 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T02:44:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T02:44:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T02:44:19.751 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000386118 s, 1.3 MB/s 2023-10-06T02:44:19.752 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T02:44:19.759 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 380 Links: 1 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2023-10-06 02:39:55.743172575 +0000 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2023-10-06 02:39:55.739172667 +0000 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2023-10-06 02:39:55.739172667 +0000 2023-10-06T02:44:19.808 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2023-10-06T02:44:19.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T02:44:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2023-10-06T02:44:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2023-10-06T02:44:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000412461 s, 1.2 MB/s 2023-10-06T02:44:19.822 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T02:44:19.830 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:19.830 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:19.830 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T02:44:19.837 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T02:44:19.837 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 02:39:54.613315975 +0000 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 02:39:54.329322508 +0000 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 02:39:54.329322508 +0000 2023-10-06T02:44:19.888 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T02:44:19.888 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T02:44:19.900 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T02:44:19.900 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T02:44:19.900 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000416118 s, 1.2 MB/s 2023-10-06T02:44:19.901 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T02:44:19.910 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2023-10-06T02:44:19.917 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T02:44:19.917 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.917 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 366 Links: 1 2023-10-06T02:44:19.918 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.918 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 02:39:55.101304748 +0000 2023-10-06T02:44:19.918 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 02:39:54.829311004 +0000 2023-10-06T02:44:19.918 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 02:39:54.829311004 +0000 2023-10-06T02:44:19.918 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T02:44:19.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T02:44:19.931 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T02:44:19.931 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T02:44:19.932 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000390758 s, 1.3 MB/s 2023-10-06T02:44:19.932 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T02:44:19.940 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2023-10-06T02:44:19.991 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 372 Links: 1 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 02:39:55.601293244 +0000 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 02:39:55.317299778 +0000 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 02:39:55.317299778 +0000 2023-10-06T02:44:19.992 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T02:44:19.992 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T02:44:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T02:44:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T02:44:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000418684 s, 1.2 MB/s 2023-10-06T02:44:20.006 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T02:44:20.014 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2023-10-06T02:44:20.019 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T02:44:20.019 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T02:44:20.019 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-10-06T02:44:20.020 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T02:44:20.020 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 02:39:56.133281004 +0000 2023-10-06T02:44:20.020 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 02:39:55.881286801 +0000 2023-10-06T02:44:20.020 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 02:39:55.881286801 +0000 2023-10-06T02:44:20.020 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T02:44:20.020 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T02:44:20.032 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T02:44:20.032 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T02:44:20.032 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000361222 s, 1.4 MB/s 2023-10-06T02:44:20.033 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T02:44:20.085 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:20.085 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T02:44:20.086 INFO:tasks.ceph:Generating config... 2023-10-06T02:44:20.087 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T02:44:20.087 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T02:44:20.087 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T02:44:20.087 INFO:tasks.ceph:[client] pid file = 2023-10-06T02:44:20.087 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T02:44:20.087 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T02:44:20.088 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T02:44:20.089 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T02:44:20.089 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T02:44:20.089 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T02:44:20.089 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T02:44:20.089 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T02:44:20.089 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T02:44:20.090 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T02:44:20.090 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T02:44:20.090 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T02:44:20.090 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T02:44:20.090 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-06T02:44:20.090 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T02:44:20.091 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T02:44:20.091 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T02:44:20.092 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T02:44:20.126 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T02:44:20.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T02:44:20.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T02:44:20.217 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.167'), ('mon.c', '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]')] 2023-10-06T02:44:20.218 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T02:44:20.219 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:20.219 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T02:44:20.224 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.167 --addv c '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:20.311 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:20.311 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid 8ff12af2-25a8-465d-8914-d29bca58b4dd 2023-10-06T02:44:20.311 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = octopus 2023-10-06T02:44:20.311 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2023-10-06T02:44:20.311 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 8ff12af2-25a8-465d-8914-d29bca58b4dd 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2023-10-06T02:44:20.308850+0000 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:created 2023-10-06T02:44:20.308850+0000 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 15 (octopus) 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2023-10-06T02:44:20.312 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2023-10-06T02:44:20.313 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T02:44:20.314 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T02:44:20.318 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8ff12af2-25a8-465d-8914-d29bca58b4dd... 2023-10-06T02:44:20.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T02:44:20.322 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T02:44:20.354 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2023-10-06T02:44:20.354 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2023-10-06T02:44:20.354 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2023-10-06T02:44:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.356 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2023-10-06T02:44:20.357 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T02:44:20.358 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T02:44:20.358 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2023-10-06T02:44:20.358 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.358 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T02:44:20.358 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.358 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2023-10-06T02:44:20.359 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2023-10-06T02:44:20.360 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 8ff12af2-25a8-465d-8914-d29bca58b4dd 2023-10-06T02:44:20.360 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.360 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2023-10-06T02:44:20.360 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2023-10-06T02:44:20.360 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.360 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.362 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2023-10-06T02:44:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2023-10-06T02:44:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2023-10-06T02:44:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2023-10-06T02:44:20.364 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2023-10-06T02:44:20.364 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.364 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2023-10-06T02:44:20.364 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2023-10-06T02:44:20.364 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2023-10-06T02:44:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2023-10-06T02:44:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 20 2023-10-06T02:44:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 20 2023-10-06T02:44:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T02:44:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2023-10-06T02:44:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 10 2023-10-06T02:44:20.366 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T02:44:20.366 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2023-10-06T02:44:20.366 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2023-10-06T02:44:20.366 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2023-10-06T02:44:20.366 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2023-10-06T02:44:20.366 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:44:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2023-10-06T02:44:20.368 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2023-10-06T02:44:20.370 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:44:20.370 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2023-10-06T02:44:20.370 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2023-10-06T02:44:20.370 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2023-10-06T02:44:20.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2023-10-06T02:44:20.372 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = "" 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2023-10-06T02:44:20.373 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2023-10-06T02:44:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.376 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.377 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2023-10-06T02:44:20.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2023-10-06T02:44:20.379 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.33,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 8ff12af2-25a8-465d-8914-d29bca58b4dd 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.380 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2023-10-06T02:44:20.381 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2023-10-06T02:44:20.381 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.381 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2023-10-06T02:44:20.381 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2023-10-06T02:44:20.381 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2023-10-06T02:44:20.381 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2023-10-06T02:44:20.382 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2023-10-06T02:44:20.383 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-10-06T02:44:20.384 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2023-10-06T02:44:20.385 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T02:44:20.386 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2023-10-06T02:44:20.387 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2023-10-06T02:44:20.388 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2023-10-06T02:44:20.389 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2023-10-06T02:44:20.390 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2023-10-06T02:44:20.391 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2023-10-06T02:44:20.391 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T02:44:20.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T02:44:20.465 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T02:44:20.466 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:20.466 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T02:44:20.473 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:20.473 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T02:44:20.479 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:44:20.479 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:20.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T02:44:20.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T02:44:20.548 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:20.548 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:20.552 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:44:20.553 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:20.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T02:44:20.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T02:44:20.581 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:20.581 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:20.585 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T02:44:20.586 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T02:44:20.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T02:44:20.632 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T02:44:20.634 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T02:44:20.680 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T02:44:20.683 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T02:44:20.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T02:44:20.728 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T02:44:20.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T02:44:20.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T02:44:20.789 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T02:44:20.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T02:44:20.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T02:44:20.897 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T02:44:20.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T02:44:20.915 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T02:44:20.950 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T02:44:20.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T02:44:20.967 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T02:44:20.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T02:44:21.003 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T02:44:21.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T02:44:21.093 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T02:44:21.106 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T02:44:21.144 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T02:44:21.157 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T02:44:21.193 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T02:44:21.206 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T02:44:21.206 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T02:44:21.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T02:44:21.224 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:21.224 INFO:tasks.ceph:role: osd.0 2023-10-06T02:44:21.224 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:44:21.224 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T02:44:21.747 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:21.748 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:21.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T02:44:21.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T02:44:21.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T02:44:21.842 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:21.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:21.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T02:44:21.854 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:21.854 INFO:tasks.ceph:role: osd.1 2023-10-06T02:44:21.854 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:44:21.854 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T02:44:22.387 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:22.388 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:22.390 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T02:44:22.390 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T02:44:22.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T02:44:22.444 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:22.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:22.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T02:44:22.455 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:22.455 INFO:tasks.ceph:role: osd.2 2023-10-06T02:44:22.455 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:44:22.455 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:22.984 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:22.985 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:22.986 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T02:44:22.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T02:44:23.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T02:44:23.055 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:23.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:23.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T02:44:23.066 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:23.066 INFO:tasks.ceph:role: osd.3 2023-10-06T02:44:23.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:44:23.066 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:23.572 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:23.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2023-10-06T02:44:23.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T02:44:23.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T02:44:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:23.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:23.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:23.757+0000 7f25ddbe43c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T02:44:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:23.757+0000 7f25ddbe43c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T02:44:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:23.757+0000 7f25ddbe43c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T02:44:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:23.757+0000 7f25ddbe43c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T02:44:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:23.757+0000 7f25ddbe43c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T02:44:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:23.765+0000 7f25ddbe43c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T02:44:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:24.593+0000 7f25ddbe43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:25.389+0000 7f25ddbe43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:26.173+0000 7f25ddbe43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:27.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T02:44:27.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.081+0000 7f27da5713c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T02:44:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.081+0000 7f27da5713c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T02:44:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.081+0000 7f27da5713c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T02:44:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.081+0000 7f27da5713c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T02:44:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.081+0000 7f27da5713c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T02:44:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.085+0000 7f27da5713c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T02:44:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:27.641+0000 7f27da5713c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:28.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:28.433+0000 7f27da5713c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:29.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:29.213+0000 7f27da5713c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:30.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T02:44:30.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.169+0000 7f541e8003c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T02:44:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.169+0000 7f541e8003c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T02:44:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.169+0000 7f541e8003c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T02:44:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.169+0000 7f541e8003c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T02:44:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.169+0000 7f541e8003c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T02:44:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.169+0000 7f541e8003c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T02:44:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:30.725+0000 7f541e8003c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:31.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:31.505+0000 7f541e8003c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:32.293+0000 7f541e8003c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:33.158 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T02:44:33.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:33.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.201+0000 7f8e4bd8c3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T02:44:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.201+0000 7f8e4bd8c3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T02:44:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.201+0000 7f8e4bd8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T02:44:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.201+0000 7f8e4bd8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T02:44:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.201+0000 7f8e4bd8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T02:44:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.205+0000 7f8e4bd8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T02:44:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:33.769+0000 7f8e4bd8c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:34.557+0000 7f8e4bd8c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:35.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:35.332+0000 7f8e4bd8c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:36.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T02:44:36.215 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T02:44:36.232 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:36.232 INFO:tasks.ceph:role: osd.4 2023-10-06T02:44:36.232 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:44:36.232 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T02:44:36.743 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:36.747 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T02:44:36.747 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T02:44:36.831 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T02:44:36.843 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:36.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:36.844 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T02:44:36.856 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:36.856 INFO:tasks.ceph:role: osd.5 2023-10-06T02:44:36.856 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:44:36.856 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:37.424 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:37.425 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:37.425 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:37.425 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:37.426 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T02:44:37.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T02:44:37.464 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T02:44:37.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:37.475 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:37.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T02:44:37.533 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:37.533 INFO:tasks.ceph:role: osd.6 2023-10-06T02:44:37.534 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:44:37.534 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T02:44:38.006 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:38.007 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:38.008 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T02:44:38.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T02:44:38.061 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T02:44:38.072 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:38.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:38.073 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T02:44:38.084 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T02:44:38.084 INFO:tasks.ceph:role: osd.7 2023-10-06T02:44:38.084 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:44:38.084 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T02:44:38.593 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T02:44:38.593 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T02:44:38.594 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T02:44:38.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-06T02:44:38.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T02:44:38.667 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T02:44:38.679 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T02:44:38.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:44:38.680 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:38.752 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:38.748+0000 7efce52c13c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T02:44:38.752 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:38.748+0000 7efce52c13c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T02:44:38.752 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:38.748+0000 7efce52c13c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T02:44:38.753 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:38.748+0000 7efce52c13c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T02:44:38.753 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:38.748+0000 7efce52c13c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T02:44:38.760 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:38.756+0000 7efce52c13c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T02:44:39.574 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:39.572+0000 7efce52c13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:40.356 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:40.352+0000 7efce52c13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:41.144 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:41.140+0000 7efce52c13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:42.026 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T02:44:42.050 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:42.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.128+0000 7fba2e5093c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T02:44:42.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.132+0000 7fba2e5093c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T02:44:42.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.132+0000 7fba2e5093c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T02:44:42.134 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.132+0000 7fba2e5093c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T02:44:42.134 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.132+0000 7fba2e5093c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T02:44:42.134 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.132+0000 7fba2e5093c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T02:44:42.696 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:42.692+0000 7fba2e5093c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:43.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:43.500+0000 7fba2e5093c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:44.294 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:44.292+0000 7fba2e5093c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:45.186 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T02:44:45.201 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:45.238 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.232+0000 7f336fce33c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T02:44:45.238 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.236+0000 7f336fce33c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T02:44:45.239 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.236+0000 7f336fce33c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T02:44:45.239 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.236+0000 7f336fce33c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T02:44:45.239 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.236+0000 7f336fce33c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T02:44:45.239 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.236+0000 7f336fce33c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T02:44:45.802 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:45.800+0000 7f336fce33c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:46.589 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:46.584+0000 7f336fce33c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:47.374 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:47.372+0000 7f336fce33c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:48.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T02:44:48.294 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:48.335 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.332+0000 7fef108e93c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T02:44:48.335 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.332+0000 7fef108e93c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T02:44:48.335 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.332+0000 7fef108e93c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T02:44:48.335 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.332+0000 7fef108e93c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T02:44:48.335 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.332+0000 7fef108e93c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T02:44:48.335 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.332+0000 7fef108e93c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T02:44:48.899 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:48.896+0000 7fef108e93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:49.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:44:49.517 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:44:49.690 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:49.688+0000 7fef108e93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:50.478 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:50.476+0000 7fef108e93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T02:44:51.333 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T02:44:51.347 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T02:44:51.347 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T02:44:51.364 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T02:44:51.421 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T02:44:51.432 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T02:44:51.443 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T02:44:51.499 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T02:44:51.513 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T02:44:51.529 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.529 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T02:44:51.546 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.546 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T02:44:51.560 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.560 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T02:44:51.572 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.572 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T02:44:51.581 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.581 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T02:44:51.639 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.640 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T02:44:51.649 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.649 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T02:44:51.659 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.660 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T02:44:51.667 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:51.667 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T02:44:51.673 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.674 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T02:44:51.707 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:51.707 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T02:44:51.714 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T02:44:51.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T02:44:51.716 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T02:44:51.727 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2023-10-06T02:44:51.727 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCEdB9l66GOJRAA5/Gl0H9QU0iET4rMvex3CQ== 2023-10-06T02:44:51.727 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2023-10-06T02:44:51.727 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCEdB9lmIpTKxAAaEJIguG5VOzRyEvP9O380Q== 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCEdB9luwLtLhAAqMTGDQaDKQ3Km6+zJNQFSA== 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCHdB9lHll+LRAAFEJKO2bT5nUkjn39YbcUPg== 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCLdB9lgBcyBRAAXbz0lU4ud1hBRoJ2iwO0Lg== 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCOdB9lUp9PChAAKWdNaHeOJKyGp81y+QRs1Q== 2023-10-06T02:44:51.728 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCRdB9ll6JZDBAAsnkvAAbpz3fQM4dJSqJrIw== 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCEdB9lLk2DKBAAwraSyfivgdCX6JLtapLX8w== 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCEdB9lwO51NRAA4LpEWhRzJs7uOlPCN/qnmw== 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCEdB9laNKfOBAAhS+9+Pn1wJFPz/vX6dZI/A== 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCWdB9lWVnKLBAAtCoJpFucjfywP/S7yJNalw== 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2023-10-06T02:44:51.729 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCadB9la1/mBxAAdmk6mMu3x9eugP16xIakpA== 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCddB9l3XIaDhAAQbwhnmEiwsQSXl37930UAQ== 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCgdB9l7y7kExAA3qsifJxMWRbTTRvY1Jqclg== 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCFdB9lylEVABAAyiDWlhvhDg77IFhO/oclZQ== 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCFdB9lXmNuBRAATJNuIDqIgXyJqPPlW6Gcng== 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2023-10-06T02:44:51.730 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCFdB9lrA+ZCBAAwUmcQFkwJkECp56B5mJ/cw== 2023-10-06T02:44:51.731 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2023-10-06T02:44:51.731 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCFdB9l4KF/CxAAE3gDNFao3CXhSdzYQ+8pEQ== 2023-10-06T02:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2023-10-06T02:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEdB9l66GOJRAA5/Gl0H9QU0iET4rMvex3CQ== 2023-10-06T02:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2023-10-06T02:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEdB9lmIpTKxAAaEJIguG5VOzRyEvP9O380Q== 2023-10-06T02:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2023-10-06T02:44:51.732 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEdB9luwLtLhAAqMTGDQaDKQ3Km6+zJNQFSA== 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCHdB9lHll+LRAAFEJKO2bT5nUkjn39YbcUPg== 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCLdB9lgBcyBRAAXbz0lU4ud1hBRoJ2iwO0Lg== 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCOdB9lUp9PChAAKWdNaHeOJKyGp81y+QRs1Q== 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2023-10-06T02:44:51.733 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCRdB9ll6JZDBAAsnkvAAbpz3fQM4dJSqJrIw== 2023-10-06T02:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2023-10-06T02:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEdB9lLk2DKBAAwraSyfivgdCX6JLtapLX8w== 2023-10-06T02:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2023-10-06T02:44:51.734 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEdB9lwO51NRAA4LpEWhRzJs7uOlPCN/qnmw== 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEdB9laNKfOBAAhS+9+Pn1wJFPz/vX6dZI/A== 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCWdB9lWVnKLBAAtCoJpFucjfywP/S7yJNalw== 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCadB9la1/mBxAAdmk6mMu3x9eugP16xIakpA== 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCddB9l3XIaDhAAQbwhnmEiwsQSXl37930UAQ== 2023-10-06T02:44:51.735 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCgdB9l7y7kExAA3qsifJxMWRbTTRvY1Jqclg== 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCFdB9lylEVABAAyiDWlhvhDg77IFhO/oclZQ== 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout:[client.2] 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCFdB9lXmNuBRAATJNuIDqIgXyJqPPlW6Gcng== 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCFdB9lrA+ZCBAAwUmcQFkwJkECp56B5mJ/cw== 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout:[client.3] 2023-10-06T02:44:51.736 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCFdB9l4KF/CxAAE3gDNFao3CXhSdzYQ+8pEQ== 2023-10-06T02:44:51.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:44:51.775 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:44:51.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:51.823 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:51.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:51.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:51.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:51.968 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.013 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.106 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.153 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:44:52.239 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T02:44:52.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:52.284 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:52.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:52.334 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T02:44:52.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.424 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.471 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.606 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T02:44:52.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.651 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.790 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.875 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T02:44:52.924 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T02:44:52.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T02:44:52.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T02:44:52.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T02:44:53.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T02:44:53.024 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T02:44:53.077 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T02:44:53.091 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T02:44:53.103 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T02:44:53.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T02:44:53.170 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:53.172 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T02:44:53.177 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T02:44:53.177 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T02:44:53.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T02:44:53.180 INFO:tasks.ceph.mon.a:Started 2023-10-06T02:44:53.180 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T02:44:53.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T02:44:53.182 INFO:tasks.ceph.mon.b:Started 2023-10-06T02:44:53.183 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T02:44:53.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T02:44:53.186 INFO:tasks.ceph.mon.c:Started 2023-10-06T02:44:53.186 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T02:44:53.186 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T02:44:53.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T02:44:53.189 INFO:tasks.ceph.mgr.y:Started 2023-10-06T02:44:53.189 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T02:44:53.189 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T02:44:53.191 INFO:tasks.ceph.mgr.x:Started 2023-10-06T02:44:53.191 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T02:44:53.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T02:44:53.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.288+0000 7f0f52fa4700 1 Processor -- start 2023-10-06T02:44:53.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.288+0000 7f0f52fa4700 1 -- start start 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.288+0000 7f0f52fa4700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.288+0000 7f0f52fa4700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c0c2080 con 0x7f0f4c0c9e60 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.288+0000 7f0f52fa4700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c0c1470 con 0x7f0f4c0c0eb0 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.288+0000 7f0f52fa4700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c0c1750 con 0x7f0f4c0c2510 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f4bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f50d40700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0f4c0c9e60 0x7f0f4c068340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41888/0 (socket says 172.21.15.33:41888) 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 learned_addr learned my addr 172.21.15.33:0/818291157 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:44:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.292+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T02:44:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.492+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.492+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.492+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.492+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.492+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T02:44:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.896+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.896+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:53.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.896+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.896+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:53.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:53.896+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T02:44:54.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:54.696+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:54.696+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:54.696+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:54.696+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:54.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:54.696+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.288+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0ca230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.288+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f4c0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.288+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 >> v1:172.21.15.167:6789/0 conn(0x7f0f4c0c0eb0 legacy=0x7f0f4c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.288+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 >> v1:172.21.15.33:6789/0 conn(0x7f0f4c0c9e60 legacy=0x7f0f4c068340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c0c1750 con 0x7f0f30003710 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30008d60 con 0x7f0f300061a0 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T02:44:56.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:56.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.292+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T02:44:56.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T02:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.492+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T02:44:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T02:44:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T02:44:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:56.892+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f4bfff700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T02:44:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T02:44:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:57.697+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T02:44:58.582 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:58.577+0000 7f5622836000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T02:44:58.583 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:58.580+0000 7f4c1309d000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T02:44:59.204 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.197+0000 7f5622836000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T02:44:59.206 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.204+0000 7f4c1309d000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T02:44:59.262 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.260+0000 7f4c1309d000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T02:44:59.263 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.257+0000 7f5622836000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T02:44:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 msgr2=0x7f0f30008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:44:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f300061a0 0x7f0f30008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 msgr2=0x7f0f30005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.167:3300/0 conn(0x7f0f30003710 0x7f0f30005b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f51541700 1 -- 172.21.15.33:0/818291157 reap_dead start 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f3000b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f300056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c0c1a30 con 0x7f0f30006600 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c0c1750 con 0x7f0f4c0c2510 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4a7fc700 1 -- 172.21.15.33:0/818291157 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30008d60 con 0x7f0f30005230 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> v2:172.21.15.33:3300/0 conn(0x7f0f4c0c2510 0x7f0f3000b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f51541700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f300056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 msgr2=0x7f0f300056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f300056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 >> v1:172.21.15.167:6789/0 conn(0x7f0f30006600 legacy=0x7f0f30003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f50d40700 1 -- 172.21.15.33:0/818291157 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30005d70 con 0x7f0f4c0c2510 2023-10-06T02:44:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4b7fe700 1 -- 172.21.15.33:0/818291157 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789246756 0 0) 0x7f0f4c0c1a30 con 0x7f0f30006600 2023-10-06T02:44:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f50d40700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f4c0c2510 0x7f0f3000b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f40007d80 tx=0x7f0f40009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbb733270248f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:44:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4b7fe700 1 -- 172.21.15.33:0/818291157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40017460 con 0x7f0f4c0c2510 2023-10-06T02:44:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4b7fe700 1 -- 172.21.15.33:0/818291157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f400175c0 con 0x7f0f4c0c2510 2023-10-06T02:44:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f4b7fe700 1 -- 172.21.15.33:0/818291157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40017940 con 0x7f0f4c0c2510 2023-10-06T02:44:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f52fa4700 1 -- 172.21.15.33:0/818291157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f4c0c2510 msgr2=0x7f0f3000b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f4c0c2510 0x7f0f3000b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f40007d80 tx=0x7f0f40009040 comp rx=0 tx=0).stop 2023-10-06T02:44:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f52fa4700 1 -- 172.21.15.33:0/818291157 shutdown_connections 2023-10-06T02:44:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f300056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.293+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/818291157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f4c0c2510 0x7f0f3000b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.297+0000 7f0f52fa4700 1 -- 172.21.15.33:0/818291157 >> 172.21.15.33:0/818291157 conn(0x7f0f4c0c83f0 msgr2=0x7f0f4c0c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:44:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.297+0000 7f0f52fa4700 1 -- 172.21.15.33:0/818291157 shutdown_connections 2023-10-06T02:44:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.297+0000 7f0f52fa4700 1 -- 172.21.15.33:0/818291157 wait complete. 2023-10-06T02:44:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.297+0000 7f0f52fa4700 1 Processor -- start 2023-10-06T02:44:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.297+0000 7f0f52fa4700 1 -- start start 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f4c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 0x7f0f4c0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f4c0c2510 0x7f0f4c0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c0adfc0 con 0x7f0f30006600 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c151bc0 con 0x7f0f4c0c2510 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c0ab4e0 con 0x7f0f30005230 2023-10-06T02:44:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 0x7f0f4c0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 0x7f0f4c0ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36828/0 (socket says 172.21.15.33:36828) 2023-10-06T02:44:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f51541700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f4c0c2510 0x7f0f4c0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f50d40700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f4c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 -- 172.21.15.33:0/3390288698 learned_addr learned my addr 172.21.15.33:0/3390288698 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 -- 172.21.15.33:0/3390288698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 msgr2=0x7f0f4c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f4c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 -- 172.21.15.33:0/3390288698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f4c0c2510 msgr2=0x7f0f4c0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f4c0c2510 0x7f0f4c0acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 -- 172.21.15.33:0/3390288698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4000f040 con 0x7f0f30006600 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f4bfff700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 0x7f0f4c0ac840 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f38017ba0 tx=0x7f0f38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3801e070 con 0x7f0f30006600 2023-10-06T02:44:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f38016ad0 con 0x7f0f30006600 2023-10-06T02:44:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c0ad3b0 con 0x7f0f30006600 2023-10-06T02:44:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.301+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38023470 con 0x7f0f30006600 2023-10-06T02:44:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.305+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c0ad970 con 0x7f0f30006600 2023-10-06T02:44:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.305+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0f380238a0 con 0x7f0f30006600 2023-10-06T02:44:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.305+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0f3802c7d0 con 0x7f0f30006600 2023-10-06T02:44:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.305+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c0710b0 con 0x7f0f30006600 2023-10-06T02:44:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.309+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0f38031060 con 0x7f0f30006600 2023-10-06T02:44:59.378 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.376+0000 7f4c1309d000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:44:59.382 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.377+0000 7f5622836000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:44:59.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.381+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0f4c0c1d10 con 0x7f0f30006600 2023-10-06T02:44:59.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f48ff9700 1 -- 172.21.15.33:0/3390288698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0f38028070 con 0x7f0f30006600 2023-10-06T02:44:59.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 msgr2=0x7f0f4c0ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 0x7f0f4c0ac840 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f38017ba0 tx=0x7f0f38008040 comp rx=0 tx=0).stop 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 shutdown_connections 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f4c0c2510 0x7f0f4c0acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f30006600 0x7f0f4c0ac840 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 --2- 172.21.15.33:0/3390288698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0f30005230 0x7f0f4c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 >> 172.21.15.33:0/3390288698 conn(0x7f0f4c0c83f0 msgr2=0x7f0f4c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 shutdown_connections 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:44:59.389+0000 7f0f52fa4700 1 -- 172.21.15.33:0/3390288698 wait complete. 2023-10-06T02:44:59.396 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2023-10-06T02:44:59.407 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T02:44:59.407 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:59.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T02:44:59.419 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:59.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T02:44:59.431 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:59.431 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T02:44:59.442 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2023-10-06T02:44:59.442 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T02:44:59.453 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:59.453 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T02:44:59.469 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:59.469 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T02:44:59.528 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:59.528 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T02:44:59.537 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T02:44:59.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T02:44:59.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 2b7ece44-9249-4d47-b37b-f1830b709f6d 0 2023-10-06T02:44:59.606 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.604+0000 7f4c1309d000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T02:44:59.623 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.617+0000 7f5622836000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T02:44:59.647 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 Processor -- start 2023-10-06T02:44:59.647 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 -- start start 2023-10-06T02:44:59.648 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f26240c8c80 0x7f26240c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.648 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2624068db0 0x7f2624069210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.648 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26240c9b70 con 0x7f26240ce2f0 2023-10-06T02:44:59.648 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240680c0 con 0x7f26240c8c80 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2629757700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240682d0 con 0x7f2624068db0 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2624068db0 0x7f2624069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2624068db0 0x7f2624069210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55284/0 (socket says 172.21.15.167:55284) 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 -- 172.21.15.167:0/2106004087 learned_addr learned my addr 172.21.15.167:0/2106004087 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2623fff700 1 --2- 172.21.15.167:0/2106004087 >> v2:172.21.15.33:3300/0 conn(0x7f26240c8c80 0x7f26240c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 -- 172.21.15.167:0/2106004087 >> v2:172.21.15.33:3300/0 conn(0x7f26240c8c80 msgr2=0x7f26240c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 --2- 172.21.15.167:0/2106004087 >> v2:172.21.15.33:3300/0 conn(0x7f26240c8c80 0x7f26240c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 -- 172.21.15.167:0/2106004087 >> v1:172.21.15.167:6789/0 conn(0x7f26240ce2f0 legacy=0x7f26240c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 -- 172.21.15.167:0/2106004087 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240685b0 con 0x7f2624068db0 2023-10-06T02:44:59.649 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f2628f56700 1 --2- 172.21.15.167:0/2106004087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2624068db0 0x7f2624069210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26240ccfc0 tx=0x7f261c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c35dd2a17c0c781 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:44:59.650 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f26237fe700 1 -- 172.21.15.167:0/2106004087 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00e070 con 0x7f2624068db0 2023-10-06T02:44:59.650 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f26237fe700 1 -- 172.21.15.167:0/2106004087 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00ba50 con 0x7f2624068db0 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.644+0000 7f26237fe700 1 -- 172.21.15.167:0/2106004087 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0092e0 con 0x7f2624068db0 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- 172.21.15.167:0/2106004087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2624068db0 msgr2=0x7f2624069210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 --2- 172.21.15.167:0/2106004087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2624068db0 0x7f2624069210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26240ccfc0 tx=0x7f261c009840 comp rx=0 tx=0).stop 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- 172.21.15.167:0/2106004087 shutdown_connections 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 --2- 172.21.15.167:0/2106004087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2624068db0 0x7f2624069210 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 --2- 172.21.15.167:0/2106004087 >> v2:172.21.15.33:3300/0 conn(0x7f26240c8c80 0x7f26240c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- 172.21.15.167:0/2106004087 >> 172.21.15.167:0/2106004087 conn(0x7f262406aab0 msgr2=0x7f26240cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- 172.21.15.167:0/2106004087 shutdown_connections 2023-10-06T02:44:59.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- 172.21.15.167:0/2106004087 wait complete. 2023-10-06T02:44:59.653 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 Processor -- start 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- start start 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 0x7f26240b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26240c8c80 0x7f26240974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26240ce2f0 0x7f2624154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624098910 con 0x7f26240c8c80 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624097a30 con 0x7f26240ce2f0 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2629757700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2624155500 con 0x7f2624068db0 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 0x7f26240b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2628f56700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26240ce2f0 0x7f2624154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 0x7f26240b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:55296/0 (socket says 172.21.15.167:55296) 2023-10-06T02:44:59.654 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 -- 172.21.15.167:0/837590018 learned_addr learned my addr 172.21.15.167:0/837590018 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f261bfff700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26240c8c80 0x7f26240974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 -- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26240ce2f0 msgr2=0x7f2624154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26240ce2f0 0x7f2624154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 -- 172.21.15.167:0/837590018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26240c8c80 msgr2=0x7f26240974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26240c8c80 0x7f26240974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f2623fff700 1 -- 172.21.15.167:0/837590018 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009ae0 con 0x7f2624068db0 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.648+0000 7f261bfff700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26240c8c80 0x7f26240974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2623fff700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 0x7f26240b2b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f261400d810 tx=0x7f2614008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26140125f0 con 0x7f2624068db0 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614012750 con 0x7f2624068db0 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2624097d00 con 0x7f2624068db0 2023-10-06T02:44:59.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614012af0 con 0x7f2624068db0 2023-10-06T02:44:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240982c0 con 0x7f2624068db0 2023-10-06T02:44:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f261401e460 con 0x7f2624068db0 2023-10-06T02:44:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2614014780 con 0x7f2624068db0 2023-10-06T02:44:59.656 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f4c1309d000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T02:44:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.652+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624071040 con 0x7f2624068db0 2023-10-06T02:44:59.657 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.656+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2614012e60 con 0x7f2624068db0 2023-10-06T02:44:59.674 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.669+0000 7f5622836000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T02:44:59.734 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.732+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2b7ece44-9249-4d47-b37b-f1830b709f6d", "id": 0} v 0) v1 -- 0x7f26240cafa0 con 0x7f2624068db0 2023-10-06T02:44:59.741 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2620ff9700 1 -- 172.21.15.167:0/837590018 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b7ece44-9249-4d47-b37b-f1830b709f6d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f261401e8a0 con 0x7f2624068db0 2023-10-06T02:44:59.741 INFO:teuthology.orchestra.run.smithi167.stdout:0 2023-10-06T02:44:59.742 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 msgr2=0x7f26240b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 0x7f26240b2b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f261400d810 tx=0x7f2614008040 comp rx=0 tx=0).stop 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 shutdown_connections 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26240ce2f0 0x7f2624154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26240c8c80 0x7f26240974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 --2- 172.21.15.167:0/837590018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2624068db0 0x7f26240b2b80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 >> 172.21.15.167:0/837590018 conn(0x7f262406aab0 msgr2=0x7f26240ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 shutdown_connections 2023-10-06T02:44:59.743 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.740+0000 7f2629757700 1 -- 172.21.15.167:0/837590018 wait complete. 2023-10-06T02:44:59.755 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5 1 2023-10-06T02:44:59.830 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.828+0000 7f4c1309d000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T02:44:59.850 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.845+0000 7f5622836000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T02:44:59.857 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 Processor -- start 2023-10-06T02:44:59.857 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- start start 2023-10-06T02:44:59.860 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 0x7fe1640aa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.860 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe164155df0 0x7fe1640b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1640aa2f0 con 0x7fe164155a20 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1640a9720 con 0x7fe164155df0 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640a99c0 con 0x7fe1640b2690 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 0x7fe1640aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 0x7fe1640aa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:55310/0 (socket says 172.21.15.167:55310) 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 -- 172.21.15.167:0/299895132 learned_addr learned my addr 172.21.15.167:0/299895132 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16ad2f700 1 --2- 172.21.15.167:0/299895132 >> v2:172.21.15.33:3300/0 conn(0x7fe164155df0 0x7fe1640b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 -- 172.21.15.167:0/299895132 >> v2:172.21.15.33:3300/0 conn(0x7fe164155df0 msgr2=0x7fe1640b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 --2- 172.21.15.167:0/299895132 >> v2:172.21.15.33:3300/0 conn(0x7fe164155df0 0x7fe1640b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.861 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 -- 172.21.15.167:0/299895132 >> v1:172.21.15.33:6789/0 conn(0x7fe164155a20 legacy=0x7fe164068ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 -- 172.21.15.167:0/299895132 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640a9ca0 con 0x7fe1640b2690 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe169d2d700 1 --2- 172.21.15.167:0/299895132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 0x7fe1640aa780 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1540099b0 tx=0x7fe154009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa4661813c69793 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16952c700 1 -- 172.21.15.167:0/299895132 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400b820 con 0x7fe1640b2690 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16952c700 1 -- 172.21.15.167:0/299895132 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15400b980 con 0x7fe1640b2690 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16952c700 1 -- 172.21.15.167:0/299895132 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400bd20 con 0x7fe1640b2690 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- 172.21.15.167:0/299895132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 msgr2=0x7fe1640aa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 --2- 172.21.15.167:0/299895132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 0x7fe1640aa780 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1540099b0 tx=0x7fe154009250 comp rx=0 tx=0).stop 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- 172.21.15.167:0/299895132 shutdown_connections 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 --2- 172.21.15.167:0/299895132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1640b2690 0x7fe1640aa780 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.862 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 --2- 172.21.15.167:0/299895132 >> v2:172.21.15.33:3300/0 conn(0x7fe164155df0 0x7fe1640b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.863 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- 172.21.15.167:0/299895132 >> 172.21.15.167:0/299895132 conn(0x7fe1640af9c0 msgr2=0x7fe1640afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:44:59.863 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- 172.21.15.167:0/299895132 shutdown_connections 2023-10-06T02:44:59.863 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.856+0000 7fe16c792700 1 -- 172.21.15.167:0/299895132 wait complete. 2023-10-06T02:44:59.865 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 Processor -- start 2023-10-06T02:44:59.865 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- start start 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1640b2690 0x7fe16414b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 0x7fe164157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe164155df0 0x7fe1640cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640af0a0 con 0x7fe1640b2690 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1641571d0 con 0x7fe164155df0 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1641565c0 con 0x7fe164155a20 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 0x7fe164157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16ad2f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe164155df0 0x7fe1640cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 0x7fe164157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:55322/0 (socket says 172.21.15.167:55322) 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 -- 172.21.15.167:0/1420185605 learned_addr learned my addr 172.21.15.167:0/1420185605 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:44:59.866 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16a52e700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1640b2690 0x7fe16414b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 -- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe164155df0 msgr2=0x7fe1640cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe164155df0 0x7fe1640cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 -- 172.21.15.167:0/1420185605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1640b2690 msgr2=0x7fe16414b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1640b2690 0x7fe16414b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 -- 172.21.15.167:0/1420185605 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540094f0 con 0x7fe164155a20 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16a52e700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1640b2690 0x7fe16414b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe169d2d700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 0x7fe164157660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe154009290 tx=0x7fe154016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154011e00 con 0x7fe164155a20 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15401b470 con 0x7fe164155a20 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1641568a0 con 0x7fe164155a20 2023-10-06T02:44:59.867 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15401b830 con 0x7fe164155a20 2023-10-06T02:44:59.868 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe164156e60 con 0x7fe164155a20 2023-10-06T02:44:59.868 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe15401c450 con 0x7fe164155a20 2023-10-06T02:44:59.868 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fe1540257b0 con 0x7fe164155a20 2023-10-06T02:44:59.868 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.860+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe164157340 con 0x7fe164155a20 2023-10-06T02:44:59.868 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.864+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe1540446d0 con 0x7fe164155a20 2023-10-06T02:44:59.892 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.888+0000 7f4c1309d000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T02:44:59.912 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.905+0000 7f5622836000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T02:44:59.942 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.940+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5", "id": 1} v 0) v1 -- 0x7fe1640710b0 con 0x7fe164155a20 2023-10-06T02:44:59.949 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe15a7fc700 1 -- 172.21.15.167:0/1420185605 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe154033070 con 0x7fe164155a20 2023-10-06T02:44:59.950 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T02:44:59.951 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 msgr2=0x7fe164157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:44:59.951 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 0x7fe164157660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe154009290 tx=0x7fe154016000 comp rx=0 tx=0).stop 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 shutdown_connections 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe164155a20 0x7fe164157660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe164155df0 0x7fe1640cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 --2- 172.21.15.167:0/1420185605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1640b2690 0x7fe16414b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 >> 172.21.15.167:0/1420185605 conn(0x7fe1640af9c0 msgr2=0x7fe16406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 shutdown_connections 2023-10-06T02:44:59.952 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:44:59.948+0000 7fe16c792700 1 -- 172.21.15.167:0/1420185605 wait complete. 2023-10-06T02:44:59.954 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:44:59.952+0000 7f4c1309d000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T02:44:59.964 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 43209fe6-de05-4f66-b8a2-cc5c6e484204 2 2023-10-06T02:44:59.976 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:44:59.969+0000 7f5622836000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T02:45:00.057 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 Processor -- start 2023-10-06T02:45:00.058 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- start start 2023-10-06T02:45:00.058 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe2080c3ac0 0x7fe2080cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe208069ba0 con 0x7fe2080ac140 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20806f530 con 0x7fe2080c3ac0 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2080ab800 con 0x7fe208068ad0 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe2080c3ac0 0x7fe2080cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe2080c3ac0 0x7fe2080cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55294/0 (socket says 172.21.15.167:55294) 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe205d9b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 -- 172.21.15.167:0/725510608 learned_addr learned my addr 172.21.15.167:0/725510608 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 -- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 msgr2=0x7fe2080cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 --2- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 -- 172.21.15.167:0/725510608 >> v1:172.21.15.167:6789/0 conn(0x7fe2080ac140 legacy=0x7fe2080c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.059 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 -- 172.21.15.167:0/725510608 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe208068f90 con 0x7fe2080c3ac0 2023-10-06T02:45:00.060 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe205d9b700 1 --2- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe206d9d700 1 --2- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080c3ac0 0x7fe2080cc3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe200009f20 tx=0x7fe200009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=413c0f9f175c2a79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20559a700 1 -- 172.21.15.167:0/725510608 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20000e070 con 0x7fe2080c3ac0 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20559a700 1 -- 172.21.15.167:0/725510608 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20000b9d0 con 0x7fe2080c3ac0 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20559a700 1 -- 172.21.15.167:0/725510608 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000092e0 con 0x7fe2080c3ac0 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080c3ac0 msgr2=0x7fe2080cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080c3ac0 0x7fe2080cc3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe200009f20 tx=0x7fe200009840 comp rx=0 tx=0).stop 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- 172.21.15.167:0/725510608 shutdown_connections 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080c3ac0 0x7fe2080cc3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.061 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/725510608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.062 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- 172.21.15.167:0/725510608 >> 172.21.15.167:0/725510608 conn(0x7fe208096ef0 msgr2=0x7fe20806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.062 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- 172.21.15.167:0/725510608 shutdown_connections 2023-10-06T02:45:00.062 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.056+0000 7fe20ce9e700 1 -- 172.21.15.167:0/725510608 wait complete. 2023-10-06T02:45:00.062 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 Processor -- start 2023-10-06T02:45:00.062 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 -- start start 2023-10-06T02:45:00.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 0x7fe2081689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2080c3ac0 0x7fe2080ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208097a90 con 0x7fe2080c3ac0 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20809b900 con 0x7fe2080ac140 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20ce9e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe208169080 con 0x7fe208068ad0 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 0x7fe2081689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 0x7fe2081689e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55310/0 (socket says 172.21.15.167:55310) 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 -- 172.21.15.167:0/2582443175 learned_addr learned my addr 172.21.15.167:0/2582443175 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20659c700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe206d9d700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2080c3ac0 0x7fe2080ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 -- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 msgr2=0x7fe2080cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 -- 172.21.15.167:0/2582443175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2080c3ac0 msgr2=0x7fe2080ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2080c3ac0 0x7fe2080ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 -- 172.21.15.167:0/2582443175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009420 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe20659c700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.060+0000 7fe205d9b700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 0x7fe2081689e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00ec80 tx=0x7fe1fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00c920 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00ca80 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00ce20 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe200009ae0 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe208169640 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe1fc013900 con 0x7fe2080ac140 2023-10-06T02:45:00.067 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fe1fc012800 con 0x7fe2080ac140 2023-10-06T02:45:00.068 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.064+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe208097c00 con 0x7fe2080ac140 2023-10-06T02:45:00.070 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.068+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe1fc021060 con 0x7fe2080ac140 2023-10-06T02:45:00.138 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.136+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "43209fe6-de05-4f66-b8a2-cc5c6e484204", "id": 2} v 0) v1 -- 0x7fe2080710b0 con 0x7fe2080ac140 2023-10-06T02:45:00.151 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe1f67fc700 1 -- 172.21.15.167:0/2582443175 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "43209fe6-de05-4f66-b8a2-cc5c6e484204", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe1fc017070 con 0x7fe2080ac140 2023-10-06T02:45:00.151 INFO:teuthology.orchestra.run.smithi167.stdout:2 2023-10-06T02:45:00.152 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 msgr2=0x7fe2081689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.152 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 0x7fe2081689e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00ec80 tx=0x7fe1fc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 shutdown_connections 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2080c3ac0 0x7fe2080ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2080ac140 0x7fe2081689e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 --2- 172.21.15.167:0/2582443175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe208068ad0 0x7fe2080cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 >> 172.21.15.167:0/2582443175 conn(0x7fe208096ef0 msgr2=0x7fe20815abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 shutdown_connections 2023-10-06T02:45:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.148+0000 7fe20ce9e700 1 -- 172.21.15.167:0/2582443175 wait complete. 2023-10-06T02:45:00.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 6ccf7bd9-d2ff-4456-ac25-6dbed4cde389 3 2023-10-06T02:45:00.268 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.264+0000 7f66c7fff700 1 Processor -- start 2023-10-06T02:45:00.269 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.264+0000 7f66c7fff700 1 -- start start 2023-10-06T02:45:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f66c80adc30 0x7f66c80af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c806a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.270 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c81575d0 con 0x7f66c80acd90 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c806ac50 con 0x7f66c80adc30 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8156fe0 con 0x7f66c806a2d0 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c806a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:55338/0 (socket says 172.21.15.167:55338) 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 -- 172.21.15.167:0/1436724267 learned_addr learned my addr 172.21.15.167:0/1436724267 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c67fc700 1 --2- 172.21.15.167:0/1436724267 >> v2:172.21.15.33:3300/0 conn(0x7f66c80adc30 0x7f66c80af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 -- 172.21.15.167:0/1436724267 >> v2:172.21.15.33:3300/0 conn(0x7f66c80adc30 msgr2=0x7f66c80af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 --2- 172.21.15.167:0/1436724267 >> v2:172.21.15.33:3300/0 conn(0x7f66c80adc30 0x7f66c80af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 -- 172.21.15.167:0/1436724267 >> v1:172.21.15.33:6789/0 conn(0x7f66c80acd90 legacy=0x7f66c8157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 -- 172.21.15.167:0/1436724267 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c80afdb0 con 0x7f66c806a2d0 2023-10-06T02:45:00.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c77fe700 1 --2- 172.21.15.167:0/1436724267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c806a710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009f20 tx=0x7f66c0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82575e9762aaede3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.272 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c5ffb700 1 -- 172.21.15.167:0/1436724267 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c000e070 con 0x7f66c806a2d0 2023-10-06T02:45:00.272 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c5ffb700 1 -- 172.21.15.167:0/1436724267 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c000ba80 con 0x7f66c806a2d0 2023-10-06T02:45:00.272 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c5ffb700 1 -- 172.21.15.167:0/1436724267 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00092e0 con 0x7f66c806a2d0 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- 172.21.15.167:0/1436724267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 msgr2=0x7f66c806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 --2- 172.21.15.167:0/1436724267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c806a710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009f20 tx=0x7f66c0009840 comp rx=0 tx=0).stop 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- 172.21.15.167:0/1436724267 shutdown_connections 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 --2- 172.21.15.167:0/1436724267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c806a710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 --2- 172.21.15.167:0/1436724267 >> v2:172.21.15.33:3300/0 conn(0x7f66c80adc30 0x7f66c80af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- 172.21.15.167:0/1436724267 >> 172.21.15.167:0/1436724267 conn(0x7f66c80cd9c0 msgr2=0x7f66c80cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- 172.21.15.167:0/1436724267 shutdown_connections 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.268+0000 7f66c7fff700 1 -- 172.21.15.167:0/1436724267 wait complete. 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 Processor -- start 2023-10-06T02:45:00.273 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- start start 2023-10-06T02:45:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c80680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.275 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66c80acd90 0x7f66c80c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 0x7f66c80c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c4d20 con 0x7f66c80acd90 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c5000 con 0x7f66c80adc30 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c52e0 con 0x7f66c806a2d0 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 0x7f66c80c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 0x7f66c80c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55312/0 (socket says 172.21.15.167:55312) 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 -- 172.21.15.167:0/773737072 learned_addr learned my addr 172.21.15.167:0/773737072 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c6ffd700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c80680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c67fc700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66c80acd90 0x7f66c80c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 -- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 msgr2=0x7f66c80680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c80680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.276 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 -- 172.21.15.167:0/773737072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66c80acd90 msgr2=0x7f66c80c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66c80acd90 0x7f66c80c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 -- 172.21.15.167:0/773737072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c0009ae0 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c77fe700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 0x7f66c80c7190 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b6c0 tx=0x7f66c0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00032f0 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c0008dd0 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c80c55c0 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0025670 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c80c49d0 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f66c0003040 con 0x7f66c80adc30 2023-10-06T02:45:00.277 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f66c0025dd0 con 0x7f66c80adc30 2023-10-06T02:45:00.278 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.272+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8071040 con 0x7f66c80adc30 2023-10-06T02:45:00.280 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.276+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f66c0020070 con 0x7f66c80adc30 2023-10-06T02:45:00.354 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.352+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ccf7bd9-d2ff-4456-ac25-6dbed4cde389", "id": 3} v 0) v1 -- 0x7f66c80b0090 con 0x7f66c80adc30 2023-10-06T02:45:00.360 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:00.356+0000 7f4c1309d000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T02:45:00.364 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.360+0000 7f66aeffd700 1 -- 172.21.15.167:0/773737072 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ccf7bd9-d2ff-4456-ac25-6dbed4cde389", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f66c001c410 con 0x7f66c80adc30 2023-10-06T02:45:00.364 INFO:teuthology.orchestra.run.smithi167.stdout:3 2023-10-06T02:45:00.366 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 msgr2=0x7f66c80c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.366 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 0x7f66c80c7190 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b6c0 tx=0x7f66c0018000 comp rx=0 tx=0).stop 2023-10-06T02:45:00.366 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 shutdown_connections 2023-10-06T02:45:00.366 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c80adc30 0x7f66c80c7190 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.367 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66c80acd90 0x7f66c80c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.367 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 --2- 172.21.15.167:0/773737072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66c806a2d0 0x7f66c80680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.367 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 >> 172.21.15.167:0/773737072 conn(0x7f66c80cd9c0 msgr2=0x7f66c8158570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.367 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 shutdown_connections 2023-10-06T02:45:00.367 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.364+0000 7f66c7fff700 1 -- 172.21.15.167:0/773737072 wait complete. 2023-10-06T02:45:00.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new e5b62b70-bdd9-43de-83ab-33415b01009c 4 2023-10-06T02:45:00.407 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:00.401+0000 7f5622836000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T02:45:00.473 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 Processor -- start 2023-10-06T02:45:00.473 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- start start 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f007815ca80 0x7f007815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00780697a0 0x7f00780b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007815ad70 con 0x7f0078159480 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780a9ca0 con 0x7f00780697a0 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780a9720 con 0x7f007815ca80 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f007815ca80 0x7f007815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f007815ca80 0x7f007815ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55324/0 (socket says 172.21.15.167:55324) 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 -- 172.21.15.167:0/187861842 learned_addr learned my addr 172.21.15.167:0/187861842 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.476 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f00767fc700 1 --2- 172.21.15.167:0/187861842 >> v2:172.21.15.33:3300/0 conn(0x7f00780697a0 0x7f00780b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 -- 172.21.15.167:0/187861842 >> v2:172.21.15.33:3300/0 conn(0x7f00780697a0 msgr2=0x7f00780b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 --2- 172.21.15.167:0/187861842 >> v2:172.21.15.33:3300/0 conn(0x7f00780697a0 0x7f00780b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 -- 172.21.15.167:0/187861842 >> v1:172.21.15.33:6789/0 conn(0x7f0078159480 legacy=0x7f00781598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 -- 172.21.15.167:0/187861842 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007815d390 con 0x7f007815ca80 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0076ffd700 1 --2- 172.21.15.167:0/187861842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f007815ca80 0x7f007815ce50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f00680099b0 tx=0x7f0068009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd85ce91416600bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0075ffb700 1 -- 172.21.15.167:0/187861842 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006800b7e0 con 0x7f007815ca80 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0075ffb700 1 -- 172.21.15.167:0/187861842 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006800b940 con 0x7f007815ca80 2023-10-06T02:45:00.477 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f0075ffb700 1 -- 172.21.15.167:0/187861842 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006800bce0 con 0x7f007815ca80 2023-10-06T02:45:00.478 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- 172.21.15.167:0/187861842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f007815ca80 msgr2=0x7f007815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.478 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 --2- 172.21.15.167:0/187861842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f007815ca80 0x7f007815ce50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f00680099b0 tx=0x7f0068009250 comp rx=0 tx=0).stop 2023-10-06T02:45:00.478 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- 172.21.15.167:0/187861842 shutdown_connections 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 --2- 172.21.15.167:0/187861842 >> v2:172.21.15.33:3300/0 conn(0x7f00780697a0 0x7f00780b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 --2- 172.21.15.167:0/187861842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f007815ca80 0x7f007815ce50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- 172.21.15.167:0/187861842 >> 172.21.15.167:0/187861842 conn(0x7f00780af9c0 msgr2=0x7f00780afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- 172.21.15.167:0/187861842 shutdown_connections 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.472+0000 7f007d6f2700 1 -- 172.21.15.167:0/187861842 wait complete. 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 Processor -- start 2023-10-06T02:45:00.479 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 -- start start 2023-10-06T02:45:00.480 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f4c1309d000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00780697a0 0x7f007814e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 0x7f0078067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815ca80 0x7f00780c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078069000 con 0x7f00780697a0 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007814e990 con 0x7f0078159480 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780684c0 con 0x7f007815ca80 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 0x7f0078067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00777fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815ca80 0x7f00780c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.482 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 0x7f0078067f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55326/0 (socket says 172.21.15.167:55326) 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 -- 172.21.15.167:0/207251946 learned_addr learned my addr 172.21.15.167:0/207251946 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f0076ffd700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00780697a0 0x7f007814e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 -- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815ca80 msgr2=0x7f00780c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815ca80 0x7f00780c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 -- 172.21.15.167:0/207251946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00780697a0 msgr2=0x7f007814e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00780697a0 0x7f007814e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 -- 172.21.15.167:0/207251946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680094f0 con 0x7f0078159480 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f00767fc700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 0x7f0078067f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f006c006b00 tx=0x7f006c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.483 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0125f0 con 0x7f0078159480 2023-10-06T02:45:00.484 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c012770 con 0x7f0078159480 2023-10-06T02:45:00.484 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c012b30 con 0x7f0078159480 2023-10-06T02:45:00.486 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0078068760 con 0x7f0078159480 2023-10-06T02:45:00.486 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.476+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0078068d20 con 0x7f0078159480 2023-10-06T02:45:00.486 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.480+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f006c01e460 con 0x7f0078159480 2023-10-06T02:45:00.486 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.480+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f006c014940 con 0x7f0078159480 2023-10-06T02:45:00.487 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.484+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007806e110 con 0x7f0078159480 2023-10-06T02:45:00.489 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.488+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f007806e110 con 0x7f0078159480 2023-10-06T02:45:00.525 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:00.517+0000 7f5622836000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T02:45:00.563 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.560+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e5b62b70-bdd9-43de-83ab-33415b01009c", "id": 4} v 0) v1 -- 0x7f00780710b0 con 0x7f0078159480 2023-10-06T02:45:00.574 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f005effd700 1 -- 172.21.15.167:0/207251946 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5b62b70-bdd9-43de-83ab-33415b01009c", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f006c014ba0 con 0x7f0078159480 2023-10-06T02:45:00.574 INFO:teuthology.orchestra.run.smithi167.stdout:4 2023-10-06T02:45:00.575 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 msgr2=0x7f0078067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.576 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 0x7f0078067f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f006c006b00 tx=0x7f006c008040 comp rx=0 tx=0).stop 2023-10-06T02:45:00.576 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 shutdown_connections 2023-10-06T02:45:00.576 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f007815ca80 0x7f00780c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.576 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0078159480 0x7f0078067f80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.577 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 --2- 172.21.15.167:0/207251946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00780697a0 0x7f007814e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.577 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 >> 172.21.15.167:0/207251946 conn(0x7f00780af9c0 msgr2=0x7f00781598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.577 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 shutdown_connections 2023-10-06T02:45:00.577 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.572+0000 7f007d6f2700 1 -- 172.21.15.167:0/207251946 wait complete. 2023-10-06T02:45:00.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55 5 2023-10-06T02:45:00.690 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ef599700 1 Processor -- start 2023-10-06T02:45:00.690 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ef599700 1 -- start start 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ef599700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f50e80cc3d0 0x7f50e806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ef599700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e8068ed0 con 0x7f50e80c7840 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ef599700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80682c0 con 0x7f50e80cc000 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ef599700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e81591f0 con 0x7f50e80cc3d0 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ee597700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f50e80cc000 0x7f50e80c7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40880/0 (socket says 172.21.15.167:40880) 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ee597700 1 -- 172.21.15.167:0/2851024370 learned_addr learned my addr 172.21.15.167:0/2851024370 (peer_addr_for_me v1:172.21.15.167:0/0) 2023-10-06T02:45:00.692 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50eed98700 1 --2- 172.21.15.167:0/2851024370 >> v2:172.21.15.33:3300/0 conn(0x7f50e80cc3d0 0x7f50e806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.694 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ed595700 1 -- 172.21.15.167:0/2851024370 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075615438 0 0) 0x7f50e80682c0 con 0x7f50e80cc000 2023-10-06T02:45:00.694 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50ed595700 1 -- 172.21.15.167:0/2851024370 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d4003570 con 0x7f50e80cc000 2023-10-06T02:45:00.694 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50eed98700 1 -- 172.21.15.167:0/2851024370 >> v1:172.21.15.167:6789/0 conn(0x7f50e80cc000 legacy=0x7f50e80c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.694 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50eed98700 1 -- 172.21.15.167:0/2851024370 >> v1:172.21.15.33:6789/0 conn(0x7f50e80c7840 legacy=0x7f50e80ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.694 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.688+0000 7f50eed98700 1 -- 172.21.15.167:0/2851024370 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e80685a0 con 0x7f50e80cc3d0 2023-10-06T02:45:00.694 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 --2- 172.21.15.167:0/2851024370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80cc3d0 0x7f50e806c3d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e60 tx=0x7f50e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1514138f5dd06d41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ed595700 1 -- 172.21.15.167:0/2851024370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000e070 con 0x7f50e80cc3d0 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ed595700 1 -- 172.21.15.167:0/2851024370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000bb60 con 0x7f50e80cc3d0 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ed595700 1 -- 172.21.15.167:0/2851024370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0012430 con 0x7f50e80cc3d0 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- 172.21.15.167:0/2851024370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80cc3d0 msgr2=0x7f50e806c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 --2- 172.21.15.167:0/2851024370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80cc3d0 0x7f50e806c3d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009e60 tx=0x7f50e0009780 comp rx=0 tx=0).stop 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- 172.21.15.167:0/2851024370 shutdown_connections 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 --2- 172.21.15.167:0/2851024370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80cc3d0 0x7f50e806c3d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- 172.21.15.167:0/2851024370 >> 172.21.15.167:0/2851024370 conn(0x7f50e80c9350 msgr2=0x7f50e80c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- 172.21.15.167:0/2851024370 shutdown_connections 2023-10-06T02:45:00.695 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- 172.21.15.167:0/2851024370 wait complete. 2023-10-06T02:45:00.697 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 Processor -- start 2023-10-06T02:45:00.697 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- start start 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80c7840 0x7f50e8159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 0x7f50e8158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80cc3d0 0x7f50e81688b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80c4080 con 0x7f50e80c7840 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80af190 con 0x7f50e80cc000 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ef599700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e8169770 con 0x7f50e80cc3d0 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50edd96700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 0x7f50e8158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50edd96700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 0x7f50e8158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55342/0 (socket says 172.21.15.167:55342) 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50edd96700 1 -- 172.21.15.167:0/2509401179 learned_addr learned my addr 172.21.15.167:0/2509401179 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80cc3d0 0x7f50e81688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50ee597700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80c7840 0x7f50e8159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.698 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 -- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 msgr2=0x7f50e8158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 0x7f50e8158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 -- 172.21.15.167:0/2509401179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80c7840 msgr2=0x7f50e8159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80c7840 0x7f50e8159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 -- 172.21.15.167:0/2509401179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009a20 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50edd96700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 0x7f50e8158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.692+0000 7f50eed98700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80cc3d0 0x7f50e81688b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50e0016000 tx=0x7f50e0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0027470 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e8169490 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0027630 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e80c3d10 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e00279b0 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f50e0024070 con 0x7f50e80cc3d0 2023-10-06T02:45:00.699 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f50e001d9a0 con 0x7f50e80cc3d0 2023-10-06T02:45:00.700 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.696+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e80c4230 con 0x7f50e80cc3d0 2023-10-06T02:45:00.702 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.700+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f50e00466d0 con 0x7f50e80cc3d0 2023-10-06T02:45:00.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.776+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55", "id": 5} v 0) v1 -- 0x7f50e8071040 con 0x7f50e80cc3d0 2023-10-06T02:45:00.780 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:00.776+0000 7f4c1309d000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T02:45:00.788 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.784+0000 7f50de7fc700 1 -- 172.21.15.167:0/2509401179 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f50e0035030 con 0x7f50e80cc3d0 2023-10-06T02:45:00.788 INFO:teuthology.orchestra.run.smithi167.stdout:5 2023-10-06T02:45:00.790 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80cc3d0 msgr2=0x7f50e81688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.790 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80cc3d0 0x7f50e81688b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50e0016000 tx=0x7f50e0019000 comp rx=0 tx=0).stop 2023-10-06T02:45:00.790 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 shutdown_connections 2023-10-06T02:45:00.790 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50e80cc3d0 0x7f50e81688b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50e80cc000 0x7f50e8158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 --2- 172.21.15.167:0/2509401179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50e80c7840 0x7f50e8159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 >> 172.21.15.167:0/2509401179 conn(0x7f50e80c9350 msgr2=0x7f50e80c1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 shutdown_connections 2023-10-06T02:45:00.791 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.788+0000 7f50ef599700 1 -- 172.21.15.167:0/2509401179 wait complete. 2023-10-06T02:45:00.805 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 7ed3b4e3-0ea4-484c-a036-f9f734aa8f08 6 2023-10-06T02:45:00.830 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:00.825+0000 7f5622836000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T02:45:00.896 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.892+0000 7f4be4f50700 1 Processor -- start 2023-10-06T02:45:00.896 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.892+0000 7f4be4f50700 1 -- start start 2023-10-06T02:45:00.900 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4be00c8c80 0x7f4be00c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.900 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00c9b70 con 0x7f4be0068db0 2023-10-06T02:45:00.900 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00682d0 con 0x7f4be00ce2f0 2023-10-06T02:45:00.900 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be00680c0 con 0x7f4be00c8c80 2023-10-06T02:45:00.900 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4be00ce2f0 0x7f4be00c9050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40882/0 (socket says 172.21.15.167:40882) 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 -- 172.21.15.167:0/3356861526 learned_addr learned my addr 172.21.15.167:0/3356861526 (peer_addr_for_me v1:172.21.15.167:0/0) 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdf7fe700 1 --2- 172.21.15.167:0/3356861526 >> v2:172.21.15.33:3300/0 conn(0x7f4be00c8c80 0x7f4be00c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143459917 0 0) 0x7f4be00682d0 con 0x7f4be00ce2f0 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bcc003570 con 0x7f4be00ce2f0 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1622904773 0 0) 0x7f4bcc003570 con 0x7f4be00ce2f0 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 >> v2:172.21.15.33:3300/0 conn(0x7f4be00c8c80 msgr2=0x7f4be00c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 --2- 172.21.15.167:0/3356861526 >> v2:172.21.15.33:3300/0 conn(0x7f4be00c8c80 0x7f4be00c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 >> v1:172.21.15.33:6789/0 conn(0x7f4be0068db0 legacy=0x7f4be0069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00685b0 con 0x7f4be00ce2f0 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 <== mon.1 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f4bc8001470 con 0x7f4be00ce2f0 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 >> v1:172.21.15.167:6789/0 conn(0x7f4be00ce2f0 legacy=0x7f4be00c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.901 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc003a60 0x7f4bcc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bcc004810 0x7f4bcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc0051c0 0x7f4bcc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00680c0 con 0x7f4bcc003a60 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc00c0d0 con 0x7f4bcc0051c0 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc00c290 con 0x7f4bcc004810 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bcc004810 0x7f4bcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdffff700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc0051c0 0x7f4bcc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdf7fe700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc003a60 0x7f4bcc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 -- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc0051c0 msgr2=0x7f4bcc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.902 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc0051c0 0x7f4bcc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 -- 172.21.15.167:0/3356861526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc003a60 msgr2=0x7f4bcc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc003a60 0x7f4bcc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 -- 172.21.15.167:0/3356861526 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc003570 con 0x7f4bcc004810 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdf7fe700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc003a60 0x7f4bcc003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdffff700 1 -- 172.21.15.167:0/3356861526 reap_dead start 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bd7fff700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bcc004810 0x7f4bcc004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c3c0 tx=0x7f4bc8001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51da1d4e4644213 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800cc90 con 0x7f4bcc004810 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc800cdf0 con 0x7f4bcc004810 2023-10-06T02:45:00.903 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4bdeffd700 1 -- 172.21.15.167:0/3356861526 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8005b50 con 0x7f4bcc004810 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bcc004810 msgr2=0x7f4bcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bcc004810 0x7f4bcc004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c3c0 tx=0x7f4bc8001e50 comp rx=0 tx=0).stop 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- 172.21.15.167:0/3356861526 shutdown_connections 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 --2- 172.21.15.167:0/3356861526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4bcc004810 0x7f4bcc004c80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- 172.21.15.167:0/3356861526 >> 172.21.15.167:0/3356861526 conn(0x7f4be006aab0 msgr2=0x7f4be00cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- 172.21.15.167:0/3356861526 shutdown_connections 2023-10-06T02:45:00.904 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.896+0000 7f4be4f50700 1 -- 172.21.15.167:0/3356861526 wait complete. 2023-10-06T02:45:00.906 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 Processor -- start 2023-10-06T02:45:00.906 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 -- start start 2023-10-06T02:45:00.906 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 0x7f4be01571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4be015b0f0 0x7f4be015eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be00b1400 0x7f4be00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be015f780 con 0x7f4be00b1400 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00b0f70 con 0x7f4bcc004810 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0157730 con 0x7f4be015b0f0 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 0x7f4be01571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 0x7f4be01571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55354/0 (socket says 172.21.15.167:55354) 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 -- 172.21.15.167:0/4069308908 learned_addr learned my addr 172.21.15.167:0/4069308908 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bd7fff700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4be015b0f0 0x7f4be015eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdffff700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be00b1400 0x7f4be00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 -- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4be015b0f0 msgr2=0x7f4be015eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.907 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4be015b0f0 0x7f4be015eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 -- 172.21.15.167:0/4069308908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be00b1400 msgr2=0x7f4be00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be00b1400 0x7f4be00b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 -- 172.21.15.167:0/4069308908 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be015f4a0 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdffff700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be00b1400 0x7f4be00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bdf7fe700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 0x7f4be01571f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0011910 tx=0x7f4bd0008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000f700 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd000f860 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be00b0920 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000fc00 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be015f070 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4bd00156e0 con 0x7f4bcc004810 2023-10-06T02:45:00.908 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.900+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f4bd001dad0 con 0x7f4bcc004810 2023-10-06T02:45:00.909 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.904+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be00b0780 con 0x7f4bcc004810 2023-10-06T02:45:00.909 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.904+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4be00b0780 con 0x7f4bcc004810 2023-10-06T02:45:00.947 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:00.944+0000 7f4c1309d000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:45:00.981 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.980+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7ed3b4e3-0ea4-484c-a036-f9f734aa8f08", "id": 6} v 0) v1 -- 0x7f4be015f200 con 0x7f4bcc004810 2023-10-06T02:45:00.992 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.988+0000 7f4bd77fe700 1 -- 172.21.15.167:0/4069308908 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ed3b4e3-0ea4-484c-a036-f9f734aa8f08", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4bd0022060 con 0x7f4bcc004810 2023-10-06T02:45:00.992 INFO:teuthology.orchestra.run.smithi167.stdout:6 2023-10-06T02:45:00.993 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 msgr2=0x7f4be01571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:00.993 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 0x7f4be01571f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0011910 tx=0x7f4bd0008040 comp rx=0 tx=0).stop 2023-10-06T02:45:00.993 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 shutdown_connections 2023-10-06T02:45:00.994 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4be015b0f0 0x7f4be015eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.994 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4be00b1400 0x7f4be00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.994 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 --2- 172.21.15.167:0/4069308908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bcc004810 0x7f4be01571f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:00.994 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 >> 172.21.15.167:0/4069308908 conn(0x7f4be006aab0 msgr2=0x7f4be00cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:00.994 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 shutdown_connections 2023-10-06T02:45:00.994 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:00.992+0000 7f4be4f50700 1 -- 172.21.15.167:0/4069308908 wait complete. 2023-10-06T02:45:01.002 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:00.997+0000 7f5622836000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T02:45:01.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new f3695573-fd9c-49c3-b8ab-93ffdc80aced 7 2023-10-06T02:45:01.013 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.012+0000 7f4c1309d000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T02:45:01.067 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.064+0000 7f4c1309d000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T02:45:01.072 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.065+0000 7f5622836000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T02:45:01.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.104+0000 7fedcce36700 1 Processor -- start 2023-10-06T02:45:01.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.104+0000 7fedcce36700 1 -- start start 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fedc81557a0 0x7fedc8159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fedc80c79c0 0x7fedc80c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc8159af0 con 0x7fedc80cb1a0 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc81594e0 con 0x7fedc81557a0 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc80c8430 con 0x7fedc80c79c0 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fedc80c79c0 0x7fedc80c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.111 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fedc80c79c0 0x7fedc80c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55360/0 (socket says 172.21.15.167:55360) 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 -- 172.21.15.167:0/1483438024 learned_addr learned my addr 172.21.15.167:0/1483438024 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedc77fe700 1 --2- 172.21.15.167:0/1483438024 >> v2:172.21.15.33:3300/0 conn(0x7fedc81557a0 0x7fedc8159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 -- 172.21.15.167:0/1483438024 >> v2:172.21.15.33:3300/0 conn(0x7fedc81557a0 msgr2=0x7fedc8159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 --2- 172.21.15.167:0/1483438024 >> v2:172.21.15.33:3300/0 conn(0x7fedc81557a0 0x7fedc8159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 -- 172.21.15.167:0/1483438024 >> v1:172.21.15.167:6789/0 conn(0x7fedc80cb1a0 legacy=0x7fedc80c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 -- 172.21.15.167:0/1483438024 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80c8710 con 0x7fedc80c79c0 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedbffff700 1 --2- 172.21.15.167:0/1483438024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc80c79c0 0x7fedc80c7e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fedc806d970 tx=0x7fedb0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f1f3f42dc68ea20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedc6ffd700 1 -- 172.21.15.167:0/1483438024 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb000b860 con 0x7fedc80c79c0 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedc6ffd700 1 -- 172.21.15.167:0/1483438024 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb000b9c0 con 0x7fedc80c79c0 2023-10-06T02:45:01.112 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedc6ffd700 1 -- 172.21.15.167:0/1483438024 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb000bd60 con 0x7fedc80c79c0 2023-10-06T02:45:01.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- 172.21.15.167:0/1483438024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc80c79c0 msgr2=0x7fedc80c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 --2- 172.21.15.167:0/1483438024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc80c79c0 0x7fedc80c7e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fedc806d970 tx=0x7fedb0009250 comp rx=0 tx=0).stop 2023-10-06T02:45:01.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- 172.21.15.167:0/1483438024 shutdown_connections 2023-10-06T02:45:01.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 --2- 172.21.15.167:0/1483438024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc80c79c0 0x7fedc80c7e40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.113 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 --2- 172.21.15.167:0/1483438024 >> v2:172.21.15.33:3300/0 conn(0x7fedc81557a0 0x7fedc8159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.114 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- 172.21.15.167:0/1483438024 >> 172.21.15.167:0/1483438024 conn(0x7fedc80c8ad0 msgr2=0x7fedc80c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:01.114 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- 172.21.15.167:0/1483438024 shutdown_connections 2023-10-06T02:45:01.114 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.108+0000 7fedcce36700 1 -- 172.21.15.167:0/1483438024 wait complete. 2023-10-06T02:45:01.114 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 Processor -- start 2023-10-06T02:45:01.114 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 -- start start 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 0x7fedc80acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc80cb1a0 0x7fedc80c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc81557a0 0x7fedc8153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80c4060 con 0x7fedc80cb1a0 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80c4300 con 0x7fedc81557a0 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80c45e0 con 0x7fedc80c79c0 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 0x7fedc80acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc81557a0 0x7fedc8153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 0x7fedc80acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:55396/0 (socket says 172.21.15.167:55396) 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc77fe700 1 -- 172.21.15.167:0/3322697306 learned_addr learned my addr 172.21.15.167:0/3322697306 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T02:45:01.117 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedbffff700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc80cb1a0 0x7fedc80c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 -- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 msgr2=0x7fedc80acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 0x7fedc80acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 -- 172.21.15.167:0/3322697306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc80cb1a0 msgr2=0x7fedc80c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc80cb1a0 0x7fedc80c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 -- 172.21.15.167:0/3322697306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00094f0 con 0x7fedc81557a0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc77fe700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 0x7fedc80acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedc7fff700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc81557a0 0x7fedc8153790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c830 tx=0x7fedc0001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0012630 con 0x7fedc81557a0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0012790 con 0x7fedc81557a0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0012b50 con 0x7fedc81557a0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc80c48c0 con 0x7fedc81557a0 2023-10-06T02:45:01.118 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc8153cd0 con 0x7fedc81557a0 2023-10-06T02:45:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fedc001b400 con 0x7fedc81557a0 2023-10-06T02:45:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.112+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fedc00119f0 con 0x7fedc81557a0 2023-10-06T02:45:01.119 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.116+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc80af0b0 con 0x7fedc81557a0 2023-10-06T02:45:01.121 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.120+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fedc80af0b0 con 0x7fedc81557a0 2023-10-06T02:45:01.124 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.120+0000 7f4c1309d000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T02:45:01.128 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.121+0000 7f5622836000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T02:45:01.187 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.181+0000 7f5622836000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T02:45:01.193 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.192+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f3695573-fd9c-49c3-b8ab-93ffdc80aced", "id": 7} v 0) v1 -- 0x7fedc8071040 con 0x7fedc81557a0 2023-10-06T02:45:01.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedbf7fe700 1 -- 172.21.15.167:0/3322697306 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f3695573-fd9c-49c3-b8ab-93ffdc80aced", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fedc0011c90 con 0x7fedc81557a0 2023-10-06T02:45:01.204 INFO:teuthology.orchestra.run.smithi167.stdout:7 2023-10-06T02:45:01.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc81557a0 msgr2=0x7fedc8153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc81557a0 0x7fedc8153790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c830 tx=0x7fedc0001690 comp rx=0 tx=0).stop 2023-10-06T02:45:01.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 shutdown_connections 2023-10-06T02:45:01.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedc81557a0 0x7fedc8153790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc80cb1a0 0x7fedc80c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 --2- 172.21.15.167:0/3322697306 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedc80c79c0 0x7fedc80acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.200+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 >> 172.21.15.167:0/3322697306 conn(0x7fedc80c8ad0 msgr2=0x7fedc8155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:01.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.204+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 shutdown_connections 2023-10-06T02:45:01.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T02:45:01.204+0000 7fedcce36700 1 -- 172.21.15.167:0/3322697306 wait complete. 2023-10-06T02:45:01.217 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T02:45:01.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T02:45:01.219 INFO:tasks.ceph.osd.0:Started 2023-10-06T02:45:01.219 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T02:45:01.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T02:45:01.221 INFO:tasks.ceph.osd.1:Started 2023-10-06T02:45:01.221 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T02:45:01.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T02:45:01.223 INFO:tasks.ceph.osd.2:Started 2023-10-06T02:45:01.223 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T02:45:01.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T02:45:01.225 INFO:tasks.ceph.osd.3:Started 2023-10-06T02:45:01.226 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T02:45:01.226 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T02:45:01.228 INFO:tasks.ceph.osd.4:Started 2023-10-06T02:45:01.228 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T02:45:01.228 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T02:45:01.229 INFO:tasks.ceph.osd.5:Started 2023-10-06T02:45:01.230 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T02:45:01.230 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T02:45:01.232 INFO:tasks.ceph.osd.6:Started 2023-10-06T02:45:01.232 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T02:45:01.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T02:45:01.234 INFO:tasks.ceph.osd.7:Started 2023-10-06T02:45:01.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:01.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 Processor -- start 2023-10-06T02:45:01.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 -- start start 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 0x7f80e0069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 0x7f80e006a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e00c7290 con 0x7f80e01554f0 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e0156570 con 0x7f80e00c7cc0 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5d67700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e0155960 con 0x7f80e0155120 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 0x7f80e0069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5566700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 0x7f80e006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 0x7f80e0069dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:34434/0 (socket says 172.21.15.33:34434) 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5566700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 0x7f80e006a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:34916/0 (socket says 172.21.15.33:34916) 2023-10-06T02:45:01.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 -- 172.21.15.33:0/3952442214 learned_addr learned my addr 172.21.15.33:0/3952442214 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 -- 172.21.15.33:0/3952442214 >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 msgr2=0x7f80e006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 --2- 172.21.15.33:0/3952442214 >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 0x7f80e006a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 -- 172.21.15.33:0/3952442214 >> v1:172.21.15.167:6789/0 conn(0x7f80e01554f0 legacy=0x7f80e00c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 -- 172.21.15.33:0/3952442214 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0155c40 con 0x7f80e0155120 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e5566700 1 --2- 172.21.15.33:0/3952442214 >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 0x7f80e006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80e4d65700 1 --2- 172.21.15.33:0/3952442214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 0x7f80e0069dc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80d40099b0 tx=0x7f80d4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a8e463d1b28c4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80df7fe700 1 -- 172.21.15.33:0/3952442214 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4009ea0 con 0x7f80e0155120 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80df7fe700 1 -- 172.21.15.33:0/3952442214 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d400b860 con 0x7f80e0155120 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.333+0000 7f80df7fe700 1 -- 172.21.15.33:0/3952442214 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d400bc00 con 0x7f80e0155120 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- 172.21.15.33:0/3952442214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 msgr2=0x7f80e0069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3952442214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 0x7f80e0069dc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80d40099b0 tx=0x7f80d4009250 comp rx=0 tx=0).stop 2023-10-06T02:45:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- 172.21.15.33:0/3952442214 shutdown_connections 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3952442214 >> v2:172.21.15.167:3300/0 conn(0x7f80e00c7cc0 0x7f80e006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3952442214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e0155120 0x7f80e0069dc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- 172.21.15.33:0/3952442214 >> 172.21.15.33:0/3952442214 conn(0x7f80e00c8ae0 msgr2=0x7f80e00c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- 172.21.15.33:0/3952442214 shutdown_connections 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- 172.21.15.33:0/3952442214 wait complete. 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 Processor -- start 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- start start 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e00c7cc0 0x7f80e014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 0x7f80e014bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e01554f0 0x7f80e014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00c43c0 con 0x7f80e0155120 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e014b9b0 con 0x7f80e00c7cc0 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5d67700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00b1550 con 0x7f80e01554f0 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 0x7f80e014bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e4d65700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e00c7cc0 0x7f80e014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 0x7f80e014bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36842/0 (socket says 172.21.15.33:36842) 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 -- 172.21.15.33:0/3268207899 learned_addr learned my addr 172.21.15.33:0/3268207899 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80e5566700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e01554f0 0x7f80e014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 -- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e01554f0 msgr2=0x7f80e014c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e01554f0 0x7f80e014c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 -- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e00c7cc0 msgr2=0x7f80e014ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e00c7cc0 0x7f80e014ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 -- 172.21.15.33:0/3268207899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8009a20 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dffff700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 0x7f80e014bf60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c590 tx=0x7f80cc007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.337+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc016070 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc006820 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc006be0 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d40094f0 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e00c3d70 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f80cc006fd0 con 0x7f80e0155120 2023-10-06T02:45:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f80cc011c70 con 0x7f80e0155120 2023-10-06T02:45:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.341+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e00c4530 con 0x7f80e0155120 2023-10-06T02:45:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.345+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f80cc007460 con 0x7f80e0155120 2023-10-06T02:45:01.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80e0071040 con 0x7f80e0155120 2023-10-06T02:45:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80dcff9700 1 -- 172.21.15.33:0/3268207899 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f80cc007600 con 0x7f80e0155120 2023-10-06T02:45:01.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:01.416 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:01.193156+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:01.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 msgr2=0x7f80e014bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 0x7f80e014bf60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c590 tx=0x7f80cc007f50 comp rx=0 tx=0).stop 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 shutdown_connections 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80e01554f0 0x7f80e014c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80e0155120 0x7f80e014bf60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.409+0000 7f80e5d67700 1 --2- 172.21.15.33:0/3268207899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e00c7cc0 0x7f80e014ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.413+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 >> 172.21.15.33:0/3268207899 conn(0x7f80e00c8ae0 msgr2=0x7f80e00c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.413+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 shutdown_connections 2023-10-06T02:45:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:01.413+0000 7f80e5d67700 1 -- 172.21.15.33:0/3268207899 wait complete. 2023-10-06T02:45:01.428 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T02:45:01.428 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T02:45:01.457 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.456+0000 7f4c1309d000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T02:45:01.505 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.501+0000 7f5622836000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T02:45:01.512 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.508+0000 7f4c1309d000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T02:45:01.563 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.557+0000 7f5622836000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T02:45:01.578 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.576+0000 7f4c1309d000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T02:45:01.583 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T02:45:01.577+0000 7f1b0db763c0 -1 Falling back to public interface 2023-10-06T02:45:01.591 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T02:45:01.585+0000 7f125e1e83c0 -1 Falling back to public interface 2023-10-06T02:45:01.627 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.621+0000 7f5622836000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T02:45:01.627 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T02:45:01.624+0000 7f3d594723c0 -1 Falling back to public interface 2023-10-06T02:45:01.634 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T02:45:01.632+0000 7f9e027e83c0 -1 Falling back to public interface 2023-10-06T02:45:01.741 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.740+0000 7f4c1309d000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T02:45:01.794 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.789+0000 7f5622836000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T02:45:01.807 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:01.804+0000 7f4c1309d000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T02:45:01.827 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T02:45:01.824+0000 7ff7d305e3c0 -1 Falling back to public interface 2023-10-06T02:45:01.842 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T02:45:01.840+0000 7f6d157513c0 -1 Falling back to public interface 2023-10-06T02:45:01.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T02:45:01.841+0000 7ffb833813c0 -1 Falling back to public interface 2023-10-06T02:45:01.847 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T02:45:01.841+0000 7fce582bb3c0 -1 Falling back to public interface 2023-10-06T02:45:01.863 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:01.857+0000 7f5622836000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T02:45:02.233 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:02.330 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32879700 1 Processor -- start 2023-10-06T02:45:02.330 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32879700 1 -- start start 2023-10-06T02:45:02.331 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32879700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a2c067c10 0x7f4a2c06e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:02.331 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32879700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2c0b1340 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32879700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2c0b1620 con 0x7f4a2c155b20 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32879700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2c0c94b0 con 0x7f4a2c067c10 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a31076700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a2c067c10 0x7f4a2c06e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32078700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4a2c0b0b00 0x7f4a2c06e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41934/0 (socket says 172.21.15.33:41934) 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a32078700 1 -- 172.21.15.33:0/310984819 learned_addr learned my addr 172.21.15.33:0/310984819 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a31076700 1 -- 172.21.15.33:0/310984819 >> v1:172.21.15.167:6789/0 conn(0x7f4a2c155b20 legacy=0x7f4a2c067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a31076700 1 -- 172.21.15.33:0/310984819 >> v1:172.21.15.33:6789/0 conn(0x7f4a2c0b0b00 legacy=0x7f4a2c06e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a31076700 1 -- 172.21.15.33:0/310984819 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0b1900 con 0x7f4a2c067c10 2023-10-06T02:45:02.332 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a31076700 1 --2- 172.21.15.33:0/310984819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c067c10 0x7f4a2c06e110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0098e0 tx=0x7f4a1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144f13d5560252b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:02.333 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a30875700 1 -- 172.21.15.33:0/310984819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c009e80 con 0x7f4a2c067c10 2023-10-06T02:45:02.333 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a30875700 1 -- 172.21.15.33:0/310984819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c00b860 con 0x7f4a2c067c10 2023-10-06T02:45:02.333 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.325+0000 7f4a30875700 1 -- 172.21.15.33:0/310984819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00bbe0 con 0x7f4a2c067c10 2023-10-06T02:45:02.333 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 -- 172.21.15.33:0/310984819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c067c10 msgr2=0x7f4a2c06e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:02.334 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 --2- 172.21.15.33:0/310984819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c067c10 0x7f4a2c06e110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0098e0 tx=0x7f4a1c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.334 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 -- 172.21.15.33:0/310984819 shutdown_connections 2023-10-06T02:45:02.334 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 --2- 172.21.15.33:0/310984819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c067c10 0x7f4a2c06e110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.334 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 -- 172.21.15.33:0/310984819 >> 172.21.15.33:0/310984819 conn(0x7f4a2c06b2d0 msgr2=0x7f4a2c06b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:02.334 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 -- 172.21.15.33:0/310984819 shutdown_connections 2023-10-06T02:45:02.334 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 -- 172.21.15.33:0/310984819 wait complete. 2023-10-06T02:45:02.336 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 Processor -- start 2023-10-06T02:45:02.336 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.329+0000 7f4a32879700 1 -- start start 2023-10-06T02:45:02.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c067c10 0x7f4a2c0c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:02.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 0x7f4a2c0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:02.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c155b20 0x7f4a2c152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0c6150 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0c34b0 con 0x7f4a2c067c10 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0c3180 con 0x7f4a2c155b20 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 0x7f4a2c0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 0x7f4a2c0c1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36866/0 (socket says 172.21.15.33:36866) 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 -- 172.21.15.33:0/2786256454 learned_addr learned my addr 172.21.15.33:0/2786256454 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32078700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c155b20 0x7f4a2c152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31877700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c067c10 0x7f4a2c0c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 -- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c155b20 msgr2=0x7f4a2c152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c155b20 0x7f4a2c152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 -- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c067c10 msgr2=0x7f4a2c0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:02.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c067c10 0x7f4a2c0c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 -- 172.21.15.33:0/2786256454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c009420 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a31076700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 0x7f4a2c0c1af0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c013000 tx=0x7f4a1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00baa0 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c025420 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2c0c23f0 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0257c0 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2c0c29b0 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4a1c023070 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f4a1c011b00 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.342 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.333+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c04be40 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.337+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4a1c02a070 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.411 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.405+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a2c05e7c0 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.412 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.405+0000 7f4a21ffb700 1 -- 172.21.15.33:0/2786256454 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f4a1c015020 con 0x7f4a2c0b0b00 2023-10-06T02:45:02.413 INFO:teuthology.misc.health.smithi033.stdout: 2023-10-06T02:45:02.413 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:01.193156+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 msgr2=0x7f4a2c0c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 0x7f4a2c0c1af0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c013000 tx=0x7f4a1c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 shutdown_connections 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a2c155b20 0x7f4a2c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a2c0b0b00 0x7f4a2c0c1af0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 --2- 172.21.15.33:0/2786256454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a2c067c10 0x7f4a2c0c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 >> 172.21.15.33:0/2786256454 conn(0x7f4a2c06b2d0 msgr2=0x7f4a2c067400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:02.414 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 shutdown_connections 2023-10-06T02:45:02.415 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:02.409+0000 7f4a32879700 1 -- 172.21.15.33:0/2786256454 wait complete. 2023-10-06T02:45:02.423 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T02:45:02.519 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:02.516+0000 7f4c1309d000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T02:45:02.603 INFO:tasks.ceph.mgr.y.smithi033.stderr:2023-10-06T02:45:02.597+0000 7f5622836000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T02:45:03.569 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:03.564+0000 7f4bfc195700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T02:45:03.569 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-06T02:45:03.568+0000 7f4bfc195700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T02:45:05.360 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T02:45:05.353+0000 7ffb833813c0 -1 osd.0 0 log_to_monitors true 2023-10-06T02:45:05.401 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T02:45:05.400+0000 7f6d157513c0 -1 osd.7 0 log_to_monitors true 2023-10-06T02:45:05.412 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T02:45:05.405+0000 7f125e1e83c0 -1 osd.2 0 log_to_monitors true 2023-10-06T02:45:05.440 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T02:45:05.433+0000 7f1b0db763c0 -1 osd.1 0 log_to_monitors true 2023-10-06T02:45:05.486 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T02:45:05.484+0000 7f3d594723c0 -1 osd.4 0 log_to_monitors true 2023-10-06T02:45:05.501 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T02:45:05.500+0000 7f9e027e83c0 -1 osd.5 0 log_to_monitors true 2023-10-06T02:45:05.610 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T02:45:05.608+0000 7ff7d305e3c0 -1 osd.6 0 log_to_monitors true 2023-10-06T02:45:05.671 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T02:45:05.665+0000 7fce582bb3c0 -1 osd.3 0 log_to_monitors true 2023-10-06T02:45:08.539 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T02:45:08.533+0000 7f1259988700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T02:45:08.549 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-10-06T02:45:08.545+0000 7f125272c700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.606 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T02:45:08.601+0000 7ffb7eb21700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T02:45:08.614 INFO:tasks.ceph.osd.0.smithi033.stderr:2023-10-06T02:45:08.609+0000 7ffb778c5700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.615 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T02:45:08.612+0000 7f6d10ef1700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T02:45:08.617 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T02:45:08.612+0000 7f3d56437700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T02:45:08.626 INFO:tasks.ceph.osd.7.smithi167.stderr:2023-10-06T02:45:08.624+0000 7f6d09c95700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.627 INFO:tasks.ceph.osd.4.smithi167.stderr:2023-10-06T02:45:08.624+0000 7f3d4d9b6700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.662 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T02:45:08.657+0000 7f1b09316700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T02:45:08.670 INFO:tasks.ceph.osd.1.smithi033.stderr:2023-10-06T02:45:08.665+0000 7f1b020ba700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.699 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T02:45:08.696+0000 7f9dfdf88700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T02:45:08.708 INFO:tasks.ceph.osd.5.smithi167.stderr:2023-10-06T02:45:08.704+0000 7f9df6d2c700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.893 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T02:45:08.889+0000 7fce53a5b700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T02:45:08.903 INFO:tasks.ceph.osd.3.smithi033.stderr:2023-10-06T02:45:08.897+0000 7fce4c7ff700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:08.979 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T02:45:08.976+0000 7ff7ce7fe700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T02:45:08.989 INFO:tasks.ceph.osd.6.smithi167.stderr:2023-10-06T02:45:08.984+0000 7ff7c75a2700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T02:45:09.230 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:09.335 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.329+0000 7fa7cb3fc700 1 Processor -- start 2023-10-06T02:45:09.335 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.329+0000 7fa7cb3fc700 1 -- start start 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7c40677b0 0x7fa7c415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c406dab0 con 0x7fa7c4154f40 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c406cea0 con 0x7fa7c4155310 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c406d180 con 0x7fa7c40677b0 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7ca3fa700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa7c4154f40 0x7fa7c406c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47310/0 (socket says 172.21.15.33:47310) 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7ca3fa700 1 -- 172.21.15.33:0/888408424 learned_addr learned my addr 172.21.15.33:0/888408424 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cabfb700 1 --2- 172.21.15.33:0/888408424 >> v2:172.21.15.33:3300/0 conn(0x7fa7c40677b0 0x7fa7c415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7c93f8700 1 -- 172.21.15.33:0/888408424 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204607407 0 0) 0x7fa7c406dab0 con 0x7fa7c4154f40 2023-10-06T02:45:09.339 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7c93f8700 1 -- 172.21.15.33:0/888408424 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b0003570 con 0x7fa7c4154f40 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cabfb700 1 -- 172.21.15.33:0/888408424 >> v1:172.21.15.167:6789/0 conn(0x7fa7c4155310 legacy=0x7fa7c40670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cabfb700 1 -- 172.21.15.33:0/888408424 >> v1:172.21.15.33:6789/0 conn(0x7fa7c4154f40 legacy=0x7fa7c406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cabfb700 1 -- 172.21.15.33:0/888408424 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c406d460 con 0x7fa7c40677b0 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cabfb700 1 --2- 172.21.15.33:0/888408424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c415fd00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00af70 tx=0x7fa7bc00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0f8cab4f7bc3c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7c93f8700 1 -- 172.21.15.33:0/888408424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00f070 con 0x7fa7c40677b0 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7c93f8700 1 -- 172.21.15.33:0/888408424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc00a2e0 con 0x7fa7c40677b0 2023-10-06T02:45:09.340 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7c93f8700 1 -- 172.21.15.33:0/888408424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00cc30 con 0x7fa7c40677b0 2023-10-06T02:45:09.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/888408424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 msgr2=0x7fa7c415fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/888408424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c415fd00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00af70 tx=0x7fa7bc00a890 comp rx=0 tx=0).stop 2023-10-06T02:45:09.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/888408424 shutdown_connections 2023-10-06T02:45:09.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/888408424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c415fd00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/888408424 >> 172.21.15.33:0/888408424 conn(0x7fa7c40c8af0 msgr2=0x7fa7c40c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:09.341 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/888408424 shutdown_connections 2023-10-06T02:45:09.342 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.333+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/888408424 wait complete. 2023-10-06T02:45:09.342 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 Processor -- start 2023-10-06T02:45:09.342 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 -- start start 2023-10-06T02:45:09.343 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c40b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c4154f40 0x7fa7c4143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4155310 0x7fa7c4157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40afe10 con 0x7fa7c40677b0 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40b02a0 con 0x7fa7c4155310 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40af200 con 0x7fa7c4154f40 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c40b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c40b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57018/0 (socket says 172.21.15.33:57018) 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 -- 172.21.15.33:0/3558066424 learned_addr learned my addr 172.21.15.33:0/3558066424 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cabfb700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4155310 0x7fa7c4157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:09.344 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7c9bf9700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c4154f40 0x7fa7c4143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:09.345 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 -- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c4154f40 msgr2=0x7fa7c4143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.345 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c4154f40 0x7fa7c4143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 -- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4155310 msgr2=0x7fa7c4157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4155310 0x7fa7c4157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 -- 172.21.15.33:0/3558066424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc00ab30 con 0x7fa7c40677b0 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cabfb700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4155310 0x7fa7c4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ca3fa700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c40b11f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c7c0 tx=0x7fa7c0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0005b30 con 0x7fa7c40677b0 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0005cb0 con 0x7fa7c40677b0 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c40af4e0 con 0x7fa7c40677b0 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0007320 con 0x7fa7c40677b0 2023-10-06T02:45:09.346 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.337+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c40afaa0 con 0x7fa7c40677b0 2023-10-06T02:45:09.347 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.341+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c404be40 con 0x7fa7c40677b0 2023-10-06T02:45:09.347 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.341+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7fa7c001b3b0 con 0x7fa7c40677b0 2023-10-06T02:45:09.348 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.341+0000 7fa7ba7fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa7a8060570 0x7fa7a8062a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:09.348 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.341+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fa7c007ae40 con 0x7fa7c40677b0 2023-10-06T02:45:09.348 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.341+0000 7fa7c9bf9700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa7a8060570 0x7fa7a8062a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:09.349 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.345+0000 7fa7c9bf9700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa7a8060570 0x7fa7a8062a20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7c414ebe0 tx=0x7fa7b4007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:09.356 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.349+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa7c004c200 con 0x7fa7c40677b0 2023-10-06T02:45:09.520 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.513+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7c406d740 con 0x7fa7c40677b0 2023-10-06T02:45:09.520 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.513+0000 7fa7ba7fc700 1 -- 172.21.15.33:0/3558066424 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fa7c004f820 con 0x7fa7c40677b0 2023-10-06T02:45:09.520 INFO:teuthology.misc.health.smithi033.stdout: 2023-10-06T02:45:09.520 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":13,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:07.574612+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:09.524 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa7a8060570 msgr2=0x7fa7a8062a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.524 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa7a8060570 0x7fa7a8062a20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7c414ebe0 tx=0x7fa7b4007ea0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 msgr2=0x7fa7c40b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c40b11f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c7c0 tx=0x7fa7c0007e00 comp rx=0 tx=0).stop 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 shutdown_connections 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa7a8060570 0x7fa7a8062a20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c4154f40 0x7fa7c4143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c4155310 0x7fa7c4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 --2- 172.21.15.33:0/3558066424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7c40677b0 0x7fa7c40b11f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 >> 172.21.15.33:0/3558066424 conn(0x7fa7c40c8af0 msgr2=0x7fa7c40ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 shutdown_connections 2023-10-06T02:45:09.525 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:09.517+0000 7fa7cb3fc700 1 -- 172.21.15.33:0/3558066424 wait complete. 2023-10-06T02:45:09.539 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T02:45:16.345 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:16.450 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 Processor -- start 2023-10-06T02:45:16.450 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- start start 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f057015d230 0x7f05700af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05700af810 0x7f0570161120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0570161660 0x7f05700cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05700afc60 con 0x7f05700af810 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0570161ae0 con 0x7f057015d230 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05700cb800 con 0x7f0570161660 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05700af810 0x7f0570161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05700af810 0x7f0570161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54246/0 (socket says 172.21.15.33:54246) 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 -- 172.21.15.33:0/722189080 learned_addr learned my addr 172.21.15.33:0/722189080 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056ffff700 1 --2- 172.21.15.33:0/722189080 >> v2:172.21.15.167:3300/0 conn(0x7f057015d230 0x7f05700af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.453 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0574c03700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0570161660 0x7f05700cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 -- 172.21.15.33:0/722189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0570161660 msgr2=0x7f05700cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0570161660 0x7f05700cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 -- 172.21.15.33:0/722189080 >> v2:172.21.15.167:3300/0 conn(0x7f057015d230 msgr2=0x7f05700af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 --2- 172.21.15.33:0/722189080 >> v2:172.21.15.167:3300/0 conn(0x7f057015d230 0x7f05700af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 -- 172.21.15.33:0/722189080 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700afe50 con 0x7f05700af810 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056ffff700 1 --2- 172.21.15.33:0/722189080 >> v2:172.21.15.167:3300/0 conn(0x7f057015d230 0x7f05700af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0574c03700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0570161660 0x7f05700cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:16.454 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056f7fe700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05700af810 0x7f0570161120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f057006b5f0 tx=0x7f0568009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d101b0204829a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056effd700 1 -- 172.21.15.33:0/722189080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800e070 con 0x7f05700af810 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056effd700 1 -- 172.21.15.33:0/722189080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056800ba40 con 0x7f05700af810 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f056effd700 1 -- 172.21.15.33:0/722189080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680092e0 con 0x7f05700af810 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- 172.21.15.33:0/722189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05700af810 msgr2=0x7f0570161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05700af810 0x7f0570161120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f057006b5f0 tx=0x7f0568009840 comp rx=0 tx=0).stop 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- 172.21.15.33:0/722189080 shutdown_connections 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0570161660 0x7f05700cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- 172.21.15.33:0/722189080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05700af810 0x7f0570161120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 --2- 172.21.15.33:0/722189080 >> v2:172.21.15.167:3300/0 conn(0x7f057015d230 0x7f05700af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- 172.21.15.33:0/722189080 >> 172.21.15.33:0/722189080 conn(0x7f05700ce6d0 msgr2=0x7f05700ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- 172.21.15.33:0/722189080 shutdown_connections 2023-10-06T02:45:16.455 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.445+0000 7f0576666700 1 -- 172.21.15.33:0/722189080 wait complete. 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 Processor -- start 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 -- start start 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05700af810 0x7f057015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 0x7f057015e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0570161660 0x7f057016dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057015ea80 con 0x7f057015d230 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057015ed60 con 0x7f0570161660 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0576666700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057015f040 con 0x7f05700af810 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 0x7f057015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.458 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05700af810 0x7f057015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f0574c03700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0570161660 0x7f057016dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 0x7f057015e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54260/0 (socket says 172.21.15.33:54260) 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056f7fe700 1 -- 172.21.15.33:0/4081636526 learned_addr learned my addr 172.21.15.33:0/4081636526 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056f7fe700 1 -- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05700af810 msgr2=0x7f057015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056f7fe700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05700af810 0x7f057015fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.449+0000 7f056f7fe700 1 -- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0570161660 msgr2=0x7f057016dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f056f7fe700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0570161660 0x7f057016dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f056f7fe700 1 -- 172.21.15.33:0/4081636526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0568009ae0 con 0x7f057015d230 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f056f7fe700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 0x7f057015e090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0568012000 tx=0x7f0568018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:16.459 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568008a00 con 0x7f057015d230 2023-10-06T02:45:16.460 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568008ba0 con 0x7f057015d230 2023-10-06T02:45:16.460 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057015f320 con 0x7f057015d230 2023-10-06T02:45:16.460 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568026460 con 0x7f057015d230 2023-10-06T02:45:16.460 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057015e5d0 con 0x7f057015d230 2023-10-06T02:45:16.461 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0568003040 con 0x7f057015d230 2023-10-06T02:45:16.461 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057015e7c0 con 0x7f057015d230 2023-10-06T02:45:16.461 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0567fff700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0550060550 0x7f0550062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.461 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.453+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f056807e2f0 con 0x7f057015d230 2023-10-06T02:45:16.461 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.457+0000 7f056ffff700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0550060550 0x7f0550062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.461 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.457+0000 7f056ffff700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0550060550 0x7f0550062a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f055800acc0 tx=0x7f0558008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:16.470 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.465+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f056804ec30 con 0x7f057015d230 2023-10-06T02:45:16.633 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0570071000 con 0x7f057015d230 2023-10-06T02:45:16.634 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0567fff700 1 -- 172.21.15.33:0/4081636526 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f0568052250 con 0x7f057015d230 2023-10-06T02:45:16.634 INFO:teuthology.misc.health.smithi033.stdout: 2023-10-06T02:45:16.634 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":18,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:13.559330+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:16.636 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0550060550 msgr2=0x7f0550062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0550060550 0x7f0550062a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f055800acc0 tx=0x7f0558008040 comp rx=0 tx=0).stop 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 msgr2=0x7f057015e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 0x7f057015e090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0568012000 tx=0x7f0568018000 comp rx=0 tx=0).stop 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 shutdown_connections 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0570161660 0x7f057016dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057015d230 0x7f057015e090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0550060550 0x7f0550062a00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 --2- 172.21.15.33:0/4081636526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05700af810 0x7f057015fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.637 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 >> 172.21.15.33:0/4081636526 conn(0x7f05700ce6d0 msgr2=0x7f057015ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:16.638 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.629+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 shutdown_connections 2023-10-06T02:45:16.638 INFO:teuthology.misc.health.smithi033.stderr:2023-10-06T02:45:16.633+0000 7f0576666700 1 -- 172.21.15.33:0/4081636526 wait complete. 2023-10-06T02:45:16.647 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T02:45:16.647 INFO:tasks.ceph:Creating RBD pool 2023-10-06T02:45:16.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T02:45:16.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.749+0000 7ff2a0b83700 1 Processor -- start 2023-10-06T02:45:16.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.749+0000 7ff2a0b83700 1 -- start start 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff2a0b83700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff29c155980 0x7ff29c0ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff2a0b83700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c15a360 con 0x7ff29c0cbc40 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff2a0b83700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c159750 con 0x7ff29c0ae970 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff2a0b83700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0c8b10 con 0x7ff29c155980 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29ad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff29c0cbc40 0x7ff29c067020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33444/0 (socket says 172.21.15.33:33444) 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29ad9d700 1 -- 172.21.15.33:0/3115533626 learned_addr learned my addr 172.21.15.33:0/3115533626 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29a59c700 1 --2- 172.21.15.33:0/3115533626 >> v2:172.21.15.167:3300/0 conn(0x7ff29c155980 0x7ff29c0ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff299d9b700 1 -- 172.21.15.33:0/3115533626 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296792299 0 0) 0x7ff29c15a360 con 0x7ff29c0cbc40 2023-10-06T02:45:16.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff299d9b700 1 -- 172.21.15.33:0/3115533626 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff288003570 con 0x7ff29c0cbc40 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff299d9b700 1 -- 172.21.15.33:0/3115533626 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217763381 0 0) 0x7ff288003570 con 0x7ff29c0cbc40 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29a59c700 1 -- 172.21.15.33:0/3115533626 >> v1:172.21.15.167:6789/0 conn(0x7ff29c0ae970 legacy=0x7ff29c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29a59c700 1 -- 172.21.15.33:0/3115533626 >> v1:172.21.15.33:6789/0 conn(0x7ff29c0cbc40 legacy=0x7ff29c067020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29a59c700 1 -- 172.21.15.33:0/3115533626 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c159a30 con 0x7ff29c155980 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff29a59c700 1 --2- 172.21.15.33:0/3115533626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c155980 0x7ff29c0ae430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0098e0 tx=0x7ff28c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4605436b6f06cdff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff299d9b700 1 -- 172.21.15.33:0/3115533626 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940017f0 con 0x7ff29c155980 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff299d9b700 1 -- 172.21.15.33:0/3115533626 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c011420 con 0x7ff29c155980 2023-10-06T02:45:16.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.753+0000 7ff299d9b700 1 -- 172.21.15.33:0/3115533626 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0117c0 con 0x7ff29c155980 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 -- 172.21.15.33:0/3115533626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c155980 msgr2=0x7ff29c0ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/3115533626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c155980 0x7ff29c0ae430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0098e0 tx=0x7ff28c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 -- 172.21.15.33:0/3115533626 shutdown_connections 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/3115533626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c155980 0x7ff29c0ae430 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 -- 172.21.15.33:0/3115533626 >> 172.21.15.33:0/3115533626 conn(0x7ff29c06a9e0 msgr2=0x7ff29c0cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 -- 172.21.15.33:0/3115533626 shutdown_connections 2023-10-06T02:45:16.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.757+0000 7ff2a0b83700 1 -- 172.21.15.33:0/3115533626 wait complete. 2023-10-06T02:45:16.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 Processor -- start 2023-10-06T02:45:16.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 -- start start 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff29c0cbc40 0x7ff29c152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 0x7ff29c0ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c0b10f0 0x7ff29c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c153390 con 0x7ff29c155980 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0b1660 con 0x7ff29c0b10f0 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c166840 con 0x7ff29c0cbc40 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 0x7ff29c0ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 0x7ff29c0ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54270/0 (socket says 172.21.15.33:54270) 2023-10-06T02:45:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 -- 172.21.15.33:0/2659331167 learned_addr learned my addr 172.21.15.33:0/2659331167 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff29a59c700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff29c0cbc40 0x7ff29c152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c0b10f0 0x7ff29c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 -- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff29c0cbc40 msgr2=0x7ff29c152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff29c0cbc40 0x7ff29c152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 -- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c0b10f0 msgr2=0x7ff29c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c0b10f0 0x7ff29c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 -- 172.21.15.33:0/2659331167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c009420 con 0x7ff29c155980 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff29a59c700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff29c0cbc40 0x7ff29c152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff29ad9d700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c0b10f0 0x7ff29c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff293fff700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 0x7ff29c0ad690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c590 tx=0x7ff28400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011590 con 0x7ff29c155980 2023-10-06T02:45:16.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2840116f0 con 0x7ff29c155980 2023-10-06T02:45:16.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c166b20 con 0x7ff29c155980 2023-10-06T02:45:16.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011ab0 con 0x7ff29c155980 2023-10-06T02:45:16.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.761+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c0adbd0 con 0x7ff29c155980 2023-10-06T02:45:16.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.765+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c0710b0 con 0x7ff29c155980 2023-10-06T02:45:16.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.765+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff28401b3b0 con 0x7ff29c155980 2023-10-06T02:45:16.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.765+0000 7ff2927fc700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff27c060550 0x7ff27c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:16.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.765+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff28407b9f0 con 0x7ff29c155980 2023-10-06T02:45:16.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.765+0000 7ff29a59c700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff27c060550 0x7ff27c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:16.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.765+0000 7ff29a59c700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff27c060550 0x7ff27c062a00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0091e0 tx=0x7ff28c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.773+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff28404c330 con 0x7ff29c155980 2023-10-06T02:45:16.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:16.937+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7ff29c159d10 con 0x7ff29c155980 2023-10-06T02:45:17.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.617+0000 7ff2927fc700 1 -- 172.21.15.33:0/2659331167 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v19) v1 ==== 109+0+0 (secure 0 0 0) 0x7ff284017020 con 0x7ff29c155980 2023-10-06T02:45:17.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff27c060550 msgr2=0x7ff27c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff27c060550 0x7ff27c062a00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0091e0 tx=0x7ff28c016000 comp rx=0 tx=0).stop 2023-10-06T02:45:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 msgr2=0x7ff29c0ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 0x7ff29c0ad690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c590 tx=0x7ff28400c950 comp rx=0 tx=0).stop 2023-10-06T02:45:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 shutdown_connections 2023-10-06T02:45:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff29c0b10f0 0x7ff29c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff29c155980 0x7ff29c0ad690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff27c060550 0x7ff27c062a00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 --2- 172.21.15.33:0/2659331167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff29c0cbc40 0x7ff29c152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 >> 172.21.15.33:0/2659331167 conn(0x7ff29c06a9e0 msgr2=0x7ff29c157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 shutdown_connections 2023-10-06T02:45:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.621+0000 7ff2a0b83700 1 -- 172.21.15.33:0/2659331167 wait complete. 2023-10-06T02:45:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'rbd' created 2023-10-06T02:45:17.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T02:45:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.753+0000 7fd6d7fff700 1 Processor -- start 2023-10-06T02:45:17.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.753+0000 7fd6d7fff700 1 -- start start 2023-10-06T02:45:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6d80cac00 0x7fd6d80cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8068420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6d81588a0 0x7fd6d8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80cb510 con 0x7fd6d81588a0 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80ce140 con 0x7fd6d80cac00 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80cd880 con 0x7fd6d80af280 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6d81588a0 0x7fd6d8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6d81588a0 0x7fd6d8158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54280/0 (socket says 172.21.15.33:54280) 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 -- 172.21.15.33:0/1586649407 learned_addr learned my addr 172.21.15.33:0/1586649407 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d67fc700 1 --2- 172.21.15.33:0/1586649407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8068420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d6ffd700 1 --2- 172.21.15.33:0/1586649407 >> v2:172.21.15.167:3300/0 conn(0x7fd6d80cac00 0x7fd6d80cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 -- 172.21.15.33:0/1586649407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 msgr2=0x7fd6d8068420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:17.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/1586649407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8068420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 -- 172.21.15.33:0/1586649407 >> v2:172.21.15.167:3300/0 conn(0x7fd6d80cac00 msgr2=0x7fd6d80cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:17.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/1586649407 >> v2:172.21.15.167:3300/0 conn(0x7fd6d80cac00 0x7fd6d80cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 -- 172.21.15.33:0/1586649407 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d80cdaf0 con 0x7fd6d81588a0 2023-10-06T02:45:17.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/1586649407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d81588a0 0x7fd6d8158d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8088a00 tx=0x7fd6d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b4090664aba99e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:17.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d5ffb700 1 -- 172.21.15.33:0/1586649407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000e070 con 0x7fd6d81588a0 2023-10-06T02:45:17.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d5ffb700 1 -- 172.21.15.33:0/1586649407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d000ba20 con 0x7fd6d81588a0 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d5ffb700 1 -- 172.21.15.33:0/1586649407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00092e0 con 0x7fd6d81588a0 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 -- 172.21.15.33:0/1586649407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d81588a0 msgr2=0x7fd6d8158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/1586649407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d81588a0 0x7fd6d8158d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8088a00 tx=0x7fd6d0009840 comp rx=0 tx=0).stop 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 -- 172.21.15.33:0/1586649407 shutdown_connections 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/1586649407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d81588a0 0x7fd6d8158d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.757+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/1586649407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8068420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.761+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/1586649407 >> v2:172.21.15.167:3300/0 conn(0x7fd6d80cac00 0x7fd6d80cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.761+0000 7fd6d7fff700 1 -- 172.21.15.33:0/1586649407 >> 172.21.15.33:0/1586649407 conn(0x7fd6d8067a30 msgr2=0x7fd6d80cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.761+0000 7fd6d7fff700 1 -- 172.21.15.33:0/1586649407 shutdown_connections 2023-10-06T02:45:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.761+0000 7fd6d7fff700 1 -- 172.21.15.33:0/1586649407 wait complete. 2023-10-06T02:45:17.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.761+0000 7fd6d7fff700 1 Processor -- start 2023-10-06T02:45:17.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.761+0000 7fd6d7fff700 1 -- start start 2023-10-06T02:45:17.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 0x7fd6d8158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d81588a0 0x7fd6d80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b36d0 con 0x7fd6d80cac00 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b39b0 con 0x7fd6d81588a0 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b3c90 con 0x7fd6d80af280 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 0x7fd6d8158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 0x7fd6d8158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54294/0 (socket says 172.21.15.33:54294) 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d67fc700 1 -- 172.21.15.33:0/113201554 learned_addr learned my addr 172.21.15.33:0/113201554 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d81588a0 0x7fd6d80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d6ffd700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 -- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 msgr2=0x7fd6d8146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:17.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 -- 172.21.15.33:0/113201554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 msgr2=0x7fd6d8158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 0x7fd6d8158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 -- 172.21.15.33:0/113201554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0009ae0 con 0x7fd6d81588a0 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d67fc700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 0x7fd6d8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d81588a0 0x7fd6d80b4770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0015000 tx=0x7fd6d0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00032f0 con 0x7fd6d81588a0 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d0008e50 con 0x7fd6d81588a0 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d80b3f70 con 0x7fd6d81588a0 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0026690 con 0x7fd6d81588a0 2023-10-06T02:45:17.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d8146b50 con 0x7fd6d81588a0 2023-10-06T02:45:17.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.765+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d80b4470 con 0x7fd6d81588a0 2023-10-06T02:45:17.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.769+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd6d0003040 con 0x7fd6d81588a0 2023-10-06T02:45:17.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.769+0000 7fd6ceffd700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6b8060550 0x7fd6b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:17.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.769+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fd6d007e5f0 con 0x7fd6d81588a0 2023-10-06T02:45:17.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.769+0000 7fd6d6ffd700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6b8060550 0x7fd6b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:17.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.769+0000 7fd6d6ffd700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6b8060550 0x7fd6b8062a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0006fd0 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:17.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.781+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6d004ede0 con 0x7fd6d81588a0 2023-10-06T02:45:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:17.949+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd6d80b3af0 con 0x7fd6d81588a0 2023-10-06T02:45:18.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.633+0000 7fd6ceffd700 1 -- 172.21.15.33:0/113201554 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v20) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd6d80b3af0 con 0x7fd6d81588a0 2023-10-06T02:45:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6b8060550 msgr2=0x7fd6b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6b8060550 0x7fd6b8062a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0006fd0 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2023-10-06T02:45:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d81588a0 msgr2=0x7fd6d80b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d81588a0 0x7fd6d80b4770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0015000 tx=0x7fd6d0018000 comp rx=0 tx=0).stop 2023-10-06T02:45:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 shutdown_connections 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d81588a0 0x7fd6d80b4770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cac00 0x7fd6d8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6b8060550 0x7fd6b8062a00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/113201554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6d80af280 0x7fd6d8146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 >> 172.21.15.33:0/113201554 conn(0x7fd6d8067a30 msgr2=0x7fd6d80b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 shutdown_connections 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.641+0000 7fd6d7fff700 1 -- 172.21.15.33:0/113201554 wait complete. 2023-10-06T02:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T02:45:18.660 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T02:45:18.660 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:45:18.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:45:18.662 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:45:18.663 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:45:18.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:45:18.665 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:45:18.665 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:45:18.665 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:45:18.667 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:45:18.667 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:45:18.667 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:45:18.669 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:45:18.669 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T02:45:18.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:18.731 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T02:45:18.733 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T02:45:18.751 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T02:45:18.761 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T02:45:18.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 Processor -- start 2023-10-06T02:45:18.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 -- start start 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5508158bf0 0x7f55080b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 0x7f55080b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55080b2610 con 0x7f5508158fc0 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5508067c10 con 0x7f5508158bf0 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.757+0000 7f550ee2e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550815d830 con 0x7f55080b0530 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 0x7f55080b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 0x7f55080b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40250/0 (socket says 172.21.15.33:40250) 2023-10-06T02:45:18.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 -- 172.21.15.33:0/2854504269 learned_addr learned my addr 172.21.15.33:0/2854504269 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:18.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550cbca700 1 --2- 172.21.15.33:0/2854504269 >> v2:172.21.15.33:3300/0 conn(0x7f5508158bf0 0x7f55080b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:18.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 -- 172.21.15.33:0/2854504269 >> v2:172.21.15.33:3300/0 conn(0x7f5508158bf0 msgr2=0x7f55080b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 --2- 172.21.15.33:0/2854504269 >> v2:172.21.15.33:3300/0 conn(0x7f5508158bf0 0x7f55080b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 -- 172.21.15.33:0/2854504269 >> v1:172.21.15.33:6789/0 conn(0x7f5508158fc0 legacy=0x7f550815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 -- 172.21.15.33:0/2854504269 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55080b00a0 con 0x7f55080b0530 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550cbca700 1 --2- 172.21.15.33:0/2854504269 >> v2:172.21.15.33:3300/0 conn(0x7f5508158bf0 0x7f55080b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f5507fff700 1 --2- 172.21.15.33:0/2854504269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 0x7f55080b20d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54f80099b0 tx=0x7f54f8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bdea755ddffbbe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f55077fe700 1 -- 172.21.15.33:0/2854504269 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f800b860 con 0x7f55080b0530 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f55077fe700 1 -- 172.21.15.33:0/2854504269 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f800b9c0 con 0x7f55080b0530 2023-10-06T02:45:18.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f55077fe700 1 -- 172.21.15.33:0/2854504269 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f800bd60 con 0x7f55080b0530 2023-10-06T02:45:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550ee2e700 1 -- 172.21.15.33:0/2854504269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 msgr2=0x7f55080b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550ee2e700 1 --2- 172.21.15.33:0/2854504269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 0x7f55080b20d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54f80099b0 tx=0x7f54f8009250 comp rx=0 tx=0).stop 2023-10-06T02:45:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550ee2e700 1 -- 172.21.15.33:0/2854504269 shutdown_connections 2023-10-06T02:45:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550ee2e700 1 --2- 172.21.15.33:0/2854504269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55080b0530 0x7f55080b20d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550ee2e700 1 --2- 172.21.15.33:0/2854504269 >> v2:172.21.15.33:3300/0 conn(0x7f5508158bf0 0x7f55080b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.761+0000 7f550ee2e700 1 -- 172.21.15.33:0/2854504269 >> 172.21.15.33:0/2854504269 conn(0x7f55080cfd70 msgr2=0x7f55080d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- 172.21.15.33:0/2854504269 shutdown_connections 2023-10-06T02:45:18.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- 172.21.15.33:0/2854504269 wait complete. 2023-10-06T02:45:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 Processor -- start 2023-10-06T02:45:18.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- start start 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5508158bf0 0x7f5508150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 0x7f55080c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55081571f0 0x7f55080cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55080c5990 con 0x7f5508158fc0 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5508159ec0 con 0x7f5508158bf0 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5508159590 con 0x7f55081571f0 2023-10-06T02:45:18.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 0x7f55080c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 0x7f55080c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54346/0 (socket says 172.21.15.33:54346) 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 -- 172.21.15.33:0/3474686176 learned_addr learned my addr 172.21.15.33:0/3474686176 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550d3cb700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55081571f0 0x7f55080cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550cbca700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5508158bf0 0x7f5508150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 -- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55081571f0 msgr2=0x7f55080cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55081571f0 0x7f55080cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 -- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5508158bf0 msgr2=0x7f5508150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5508158bf0 0x7f5508150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 -- 172.21.15.33:0/3474686176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80094f0 con 0x7f5508158fc0 2023-10-06T02:45:18.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5507fff700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 0x7f55080c5030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009290 tx=0x7f54f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:18.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f8011e40 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.765+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5508159870 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f801a450 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081506a0 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f801a7d0 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f54f801ab40 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55080aefe0 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f5504ff9700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f54e8060580 0x7f54e8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f54f807de60 con 0x7f5508158fc0 2023-10-06T02:45:18.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f550cbca700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f54e8060580 0x7f54e8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:18.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.769+0000 7f550cbca700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f54e8060580 0x7f54e8062a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5500007740 tx=0x7f55000099d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:18.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.773+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f55080aefe0 con 0x7f5508158fc0 2023-10-06T02:45:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.933+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5508067d60 con 0x7f5508158fc0 2023-10-06T02:45:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f5504ff9700 1 -- 172.21.15.33:0/3474686176 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f5508067d60 con 0x7f5508158fc0 2023-10-06T02:45:18.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:18.942 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:18.616675+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f54e8060580 msgr2=0x7f54e8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f54e8060580 0x7f54e8062a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5500007740 tx=0x7f55000099d0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 msgr2=0x7f55080c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 0x7f55080c5030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009290 tx=0x7f54f8017000 comp rx=0 tx=0).stop 2023-10-06T02:45:18.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 shutdown_connections 2023-10-06T02:45:18.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f54e8060580 0x7f54e8062a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55081571f0 0x7f55080cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5508158fc0 0x7f55080c5030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 --2- 172.21.15.33:0/3474686176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5508158bf0 0x7f5508150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:18.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 >> 172.21.15.33:0/3474686176 conn(0x7f55080cfd70 msgr2=0x7f55080ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:18.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 shutdown_connections 2023-10-06T02:45:18.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:18.937+0000 7f550ee2e700 1 -- 172.21.15.33:0/3474686176 wait complete. 2023-10-06T02:45:18.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:45:18.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:45:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.069+0000 7fd6e6f0e700 1 Processor -- start 2023-10-06T02:45:19.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.069+0000 7fd6e6f0e700 1 -- start start 2023-10-06T02:45:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6e00cace0 0x7fd6e00c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00c9d80 con 0x7fd6e015a1e0 2023-10-06T02:45:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00ca060 con 0x7fd6e00cc900 2023-10-06T02:45:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00c29a0 con 0x7fd6e00cace0 2023-10-06T02:45:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6e00cace0 0x7fd6e00c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e4caa700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd6e015a1e0 0x7fd6e00ca5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33468/0 (socket says 172.21.15.33:33468) 2023-10-06T02:45:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e4caa700 1 -- 172.21.15.33:0/498536157 learned_addr learned my addr 172.21.15.33:0/498536157 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/498536157 >> v2:172.21.15.33:3300/0 conn(0x7fd6e00cace0 0x7fd6e00c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54362/0 (socket says 172.21.15.33:54362) 2023-10-06T02:45:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d7fff700 1 -- 172.21.15.33:0/498536157 >> v1:172.21.15.167:6789/0 conn(0x7fd6e00cc900 legacy=0x7fd6e00c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d7fff700 1 -- 172.21.15.33:0/498536157 >> v1:172.21.15.33:6789/0 conn(0x7fd6e015a1e0 legacy=0x7fd6e00ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d7fff700 1 -- 172.21.15.33:0/498536157 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00c2c80 con 0x7fd6e00cace0 2023-10-06T02:45:19.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/498536157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e00c8790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00098e0 tx=0x7fd6d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85b5c5deaaf870e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d77fe700 1 -- 172.21.15.33:0/498536157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0009e80 con 0x7fd6e00cace0 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d77fe700 1 -- 172.21.15.33:0/498536157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d000b860 con 0x7fd6e00cace0 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6d77fe700 1 -- 172.21.15.33:0/498536157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000bbe0 con 0x7fd6e00cace0 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/498536157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 msgr2=0x7fd6e00c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/498536157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e00c8790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00098e0 tx=0x7fd6d00091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/498536157 shutdown_connections 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/498536157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e00c8790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/498536157 >> 172.21.15.33:0/498536157 conn(0x7fd6e0096ef0 msgr2=0x7fd6e006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/498536157 shutdown_connections 2023-10-06T02:45:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/498536157 wait complete. 2023-10-06T02:45:19.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 Processor -- start 2023-10-06T02:45:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.073+0000 7fd6e6f0e700 1 -- start start 2023-10-06T02:45:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e0169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.086 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e00cc900 0x7fd6e015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e015a1e0 0x7fd6e0147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0169a20 con 0x7fd6e00cace0 2023-10-06T02:45:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0069f70 con 0x7fd6e015a1e0 2023-10-06T02:45:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0147bd0 con 0x7fd6e00cc900 2023-10-06T02:45:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e0169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e0169eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54372/0 (socket says 172.21.15.33:54372) 2023-10-06T02:45:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 -- 172.21.15.33:0/4781890 learned_addr learned my addr 172.21.15.33:0/4781890 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e00cc900 0x7fd6e015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e54ab700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e015a1e0 0x7fd6e0147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 -- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e00cc900 msgr2=0x7fd6e015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e00cc900 0x7fd6e015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 -- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e015a1e0 msgr2=0x7fd6e0147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e015a1e0 0x7fd6e0147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 -- 172.21.15.33:0/4781890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0009420 con 0x7fd6e00cace0 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e4caa700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e0169eb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00c5c0 tx=0x7fd6dc001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc005b30 con 0x7fd6e00cace0 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc005cb0 con 0x7fd6e00cace0 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e015c810 con 0x7fd6e00cace0 2023-10-06T02:45:19.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc007320 con 0x7fd6e00cace0 2023-10-06T02:45:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.077+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e01693d0 con 0x7fd6e00cace0 2023-10-06T02:45:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.081+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd6dc01b3b0 con 0x7fd6e00cace0 2023-10-06T02:45:19.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.081+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e015aaf0 con 0x7fd6e00cace0 2023-10-06T02:45:19.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.081+0000 7fd6d4ff9700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6cc060550 0x7fd6cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.081+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd6dc07b9c0 con 0x7fd6e00cace0 2023-10-06T02:45:19.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.081+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6cc060550 0x7fd6cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.081+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6cc060550 0x7fd6cc062a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00cb0e0 tx=0x7fd6d0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.095 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.089+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6dc04c100 con 0x7fd6e00cace0 2023-10-06T02:45:19.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.253+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6e00710b0 con 0x7fd6e00cace0 2023-10-06T02:45:19.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.253+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/4781890 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6dc04f720 con 0x7fd6e00cace0 2023-10-06T02:45:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6cc060550 msgr2=0x7fd6cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6cc060550 0x7fd6cc062a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00cb0e0 tx=0x7fd6d0017000 comp rx=0 tx=0).stop 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 msgr2=0x7fd6e0169eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e0169eb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00c5c0 tx=0x7fd6dc001020 comp rx=0 tx=0).stop 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 shutdown_connections 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6cc060550 0x7fd6cc062a00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6e00cc900 0x7fd6e015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6e015a1e0 0x7fd6e0147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 --2- 172.21.15.33:0/4781890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e00cace0 0x7fd6e0169eb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 >> 172.21.15.33:0/4781890 conn(0x7fd6e0096ef0 msgr2=0x7fd6e01535b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 shutdown_connections 2023-10-06T02:45:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.257+0000 7fd6e6f0e700 1 -- 172.21.15.33:0/4781890 wait complete. 2023-10-06T02:45:19.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:45:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab54f4f700 1 Processor -- start 2023-10-06T02:45:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab54f4f700 1 -- start start 2023-10-06T02:45:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab54f4f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab500cf7c0 0x7fab500b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab54f4f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab500ce580 con 0x7fab50068aa0 2023-10-06T02:45:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab54f4f700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab500ce860 con 0x7fab50158c60 2023-10-06T02:45:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab54f4f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab500cf620 con 0x7fab500cf7c0 2023-10-06T02:45:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab4ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab500cf7c0 0x7fab500b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab46ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab50068aa0 0x7fab500b29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33472/0 (socket says 172.21.15.33:33472) 2023-10-06T02:45:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.385+0000 7fab46ffd700 1 -- 172.21.15.33:0/1341519807 learned_addr learned my addr 172.21.15.33:0/1341519807 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:19.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4ffff700 1 -- 172.21.15.33:0/1341519807 >> v1:172.21.15.167:6789/0 conn(0x7fab50158c60 legacy=0x7fab50068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4ffff700 1 -- 172.21.15.33:0/1341519807 >> v1:172.21.15.33:6789/0 conn(0x7fab50068aa0 legacy=0x7fab500b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4ffff700 1 -- 172.21.15.33:0/1341519807 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500ceb40 con 0x7fab500cf7c0 2023-10-06T02:45:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4ffff700 1 --2- 172.21.15.33:0/1341519807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab500cf7c0 0x7fab500b30f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab48009e60 tx=0x7fab48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3746f597ca3081f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4effd700 1 -- 172.21.15.33:0/1341519807 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800e070 con 0x7fab500cf7c0 2023-10-06T02:45:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4effd700 1 -- 172.21.15.33:0/1341519807 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800bc30 con 0x7fab500cf7c0 2023-10-06T02:45:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab4effd700 1 -- 172.21.15.33:0/1341519807 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48012430 con 0x7fab500cf7c0 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 -- 172.21.15.33:0/1341519807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab500cf7c0 msgr2=0x7fab500b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 --2- 172.21.15.33:0/1341519807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab500cf7c0 0x7fab500b30f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab48009e60 tx=0x7fab48009780 comp rx=0 tx=0).stop 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 -- 172.21.15.33:0/1341519807 shutdown_connections 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 --2- 172.21.15.33:0/1341519807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab500cf7c0 0x7fab500b30f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 -- 172.21.15.33:0/1341519807 >> 172.21.15.33:0/1341519807 conn(0x7fab50067a40 msgr2=0x7fab500b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 -- 172.21.15.33:0/1341519807 shutdown_connections 2023-10-06T02:45:19.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.389+0000 7fab54f4f700 1 -- 172.21.15.33:0/1341519807 wait complete. 2023-10-06T02:45:19.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 Processor -- start 2023-10-06T02:45:19.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 -- start start 2023-10-06T02:45:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 0x7fab5015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab500cf7c0 0x7fab50153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab50158c60 0x7fab5016c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015d940 con 0x7fab50068aa0 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500a6910 con 0x7fab500cf7c0 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab54f4f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab500adbc0 con 0x7fab50158c60 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab4f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 0x7fab5015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab4f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 0x7fab5015d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54394/0 (socket says 172.21.15.33:54394) 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab4f7fe700 1 -- 172.21.15.33:0/3195548338 learned_addr learned my addr 172.21.15.33:0/3195548338 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab4ffff700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab50158c60 0x7fab5016c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.393+0000 7fab46ffd700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab500cf7c0 0x7fab50153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab4f7fe700 1 -- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab50158c60 msgr2=0x7fab5016c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab4f7fe700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab50158c60 0x7fab5016c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab4f7fe700 1 -- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab500cf7c0 msgr2=0x7fab50153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab4f7fe700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab500cf7c0 0x7fab50153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab4f7fe700 1 -- 172.21.15.33:0/3195548338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009a20 con 0x7fab50068aa0 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab4f7fe700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 0x7fab5015d2b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c4c0 tx=0x7fab4000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000cea0 con 0x7fab50068aa0 2023-10-06T02:45:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40011590 con 0x7fab50068aa0 2023-10-06T02:45:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab500adea0 con 0x7fab50068aa0 2023-10-06T02:45:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40011930 con 0x7fab50068aa0 2023-10-06T02:45:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5016d120 con 0x7fab50068aa0 2023-10-06T02:45:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.397+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50071040 con 0x7fab50068aa0 2023-10-06T02:45:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.401+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fab40011ce0 con 0x7fab50068aa0 2023-10-06T02:45:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.401+0000 7fab47fff700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab34060550 0x7fab34062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.401+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fab4007bc30 con 0x7fab50068aa0 2023-10-06T02:45:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.401+0000 7fab46ffd700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab34060550 0x7fab34062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.401+0000 7fab46ffd700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab34060550 0x7fab34062a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab5015c120 tx=0x7fab38007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.401+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fab4004c540 con 0x7fab50068aa0 2023-10-06T02:45:19.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.565+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fab500cee20 con 0x7fab50068aa0 2023-10-06T02:45:19.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.565+0000 7fab47fff700 1 -- 172.21.15.33:0/3195548338 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fab40017070 con 0x7fab50068aa0 2023-10-06T02:45:19.572 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:45:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab34060550 msgr2=0x7fab34062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab34060550 0x7fab34062a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab5015c120 tx=0x7fab38007eb0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 msgr2=0x7fab5015d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 0x7fab5015d2b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c4c0 tx=0x7fab4000c7d0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 shutdown_connections 2023-10-06T02:45:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab34060550 0x7fab34062a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab50158c60 0x7fab5016c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab500cf7c0 0x7fab50153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 --2- 172.21.15.33:0/3195548338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab50068aa0 0x7fab5015d2b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 >> 172.21.15.33:0/3195548338 conn(0x7fab50067a40 msgr2=0x7fab500b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 shutdown_connections 2023-10-06T02:45:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.569+0000 7fab54f4f700 1 -- 172.21.15.33:0/3195548338 wait complete. 2023-10-06T02:45:19.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:19.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:45:19.653 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.685+0000 7f1045dfa700 1 Processor -- start 2023-10-06T02:45:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.685+0000 7f1045dfa700 1 -- start start 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1040067fc0 0x7f1040068400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1040068940 con 0x7f1040156d80 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10400ca260 con 0x7f10401569b0 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10400ca540 con 0x7f1040067fc0 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1040156d80 0x7f10400a9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33474/0 (socket says 172.21.15.33:33474) 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103effd700 1 -- 172.21.15.33:0/2640492364 learned_addr learned my addr 172.21.15.33:0/2640492364 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103ffff700 1 --2- 172.21.15.33:0/2640492364 >> v2:172.21.15.167:3300/0 conn(0x7f1040067fc0 0x7f1040068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232086764 0 0) 0x7f1040068940 con 0x7f1040156d80 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f102c003570 con 0x7f1040156d80 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2764402892 0 0) 0x7f102c003570 con 0x7f1040156d80 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 >> v2:172.21.15.167:3300/0 conn(0x7f1040067fc0 msgr2=0x7f1040068400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 --2- 172.21.15.33:0/2640492364 >> v2:172.21.15.167:3300/0 conn(0x7f1040067fc0 0x7f1040068400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 >> v1:172.21.15.167:6789/0 conn(0x7f10401569b0 legacy=0x7f1040069ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400ca820 con 0x7f1040156d80 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f1028001470 con 0x7f1040156d80 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 >> v1:172.21.15.33:6789/0 conn(0x7f1040156d80 legacy=0x7f10400a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f102c003a90 0x7f102c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c004810 0x7f102c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c0051c0 0x7f102c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400ca540 con 0x7f102c003a90 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c00c080 con 0x7f102c0051c0 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102c00c240 con 0x7f102c004810 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f102c003a90 0x7f102c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103ffff700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c0051c0 0x7f102c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103effd700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c004810 0x7f102c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 -- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c004810 msgr2=0x7f102c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c004810 0x7f102c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 -- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c0051c0 msgr2=0x7f102c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c0051c0 0x7f102c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103ffff700 1 -- 172.21.15.33:0/2640492364 reap_dead start 2023-10-06T02:45:19.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 -- 172.21.15.33:0/2640492364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c003570 con 0x7f102c003a90 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f102c003a90 0x7f102c003e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1038002070 tx=0x7f10380016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b50e5556e9989 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380019e0 con 0x7f102c003a90 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103800c8b0 con 0x7f102c003a90 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f103e7fc700 1 -- 172.21.15.33:0/2640492364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103800cc50 con 0x7f102c003a90 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- 172.21.15.33:0/2640492364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f102c003a90 msgr2=0x7f102c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f102c003a90 0x7f102c003e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1038002070 tx=0x7f10380016d0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- 172.21.15.33:0/2640492364 shutdown_connections 2023-10-06T02:45:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2640492364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f102c003a90 0x7f102c003e60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- 172.21.15.33:0/2640492364 >> 172.21.15.33:0/2640492364 conn(0x7f10400c97b0 msgr2=0x7f104006e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- 172.21.15.33:0/2640492364 shutdown_connections 2023-10-06T02:45:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.689+0000 7f1045dfa700 1 -- 172.21.15.33:0/2640492364 wait complete. 2023-10-06T02:45:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 Processor -- start 2023-10-06T02:45:19.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 -- start start 2023-10-06T02:45:19.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c003a90 0x7f104014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 0x7f10400c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400c53e0 0x7f10400c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c5d90 con 0x7f10400c4a50 2023-10-06T02:45:19.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040157280 con 0x7f102c003a90 2023-10-06T02:45:19.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f1045dfa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400b0ad0 con 0x7f10400c53e0 2023-10-06T02:45:19.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 0x7f10400c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 0x7f10400c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54416/0 (socket says 172.21.15.33:54416) 2023-10-06T02:45:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 -- 172.21.15.33:0/2024077033 learned_addr learned my addr 172.21.15.33:0/2024077033 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103ffff700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400c53e0 0x7f10400c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c003a90 0x7f104014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 -- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400c53e0 msgr2=0x7f10400c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400c53e0 0x7f10400c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 -- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c003a90 msgr2=0x7f104014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c003a90 0x7f104014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 -- 172.21.15.33:0/2024077033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400b1610 con 0x7f10400c4a50 2023-10-06T02:45:19.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.693+0000 7f103effd700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 0x7f10400c4ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f102800e810 tx=0x7f1028001010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800c5a0 con 0x7f10400c4a50 2023-10-06T02:45:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102800c700 con 0x7f10400c4a50 2023-10-06T02:45:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800cac0 con 0x7f10400c4a50 2023-10-06T02:45:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10400c5830 con 0x7f10400c4a50 2023-10-06T02:45:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10400b1050 con 0x7f10400c4a50 2023-10-06T02:45:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10400710b0 con 0x7f10400c4a50 2023-10-06T02:45:19.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f102801d460 con 0x7f10400c4a50 2023-10-06T02:45:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f10377fe700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1020060550 0x7f1020062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f102807c470 con 0x7f10400c4a50 2023-10-06T02:45:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.697+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1020060550 0x7f1020062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.701+0000 7f103f7fe700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1020060550 0x7f1020062a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1038002070 tx=0x7f1038015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.705+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f102804cc50 con 0x7f10400c4a50 2023-10-06T02:45:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f104006ed60 con 0x7f10400c4a50 2023-10-06T02:45:19.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f10377fe700 1 -- 172.21.15.33:0/2024077033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f1028050270 con 0x7f10400c4a50 2023-10-06T02:45:19.915 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:19.915 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T02:45:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1020060550 msgr2=0x7f1020062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1020060550 0x7f1020062a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1038002070 tx=0x7f1038015000 comp rx=0 tx=0).stop 2023-10-06T02:45:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 msgr2=0x7f10400c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 0x7f10400c4ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f102800e810 tx=0x7f1028001010 comp rx=0 tx=0).stop 2023-10-06T02:45:19.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 shutdown_connections 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1020060550 0x7f1020062a00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.909+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400c53e0 0x7f10400c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.913+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10400c4a50 0x7f10400c4ea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.913+0000 7f1045dfa700 1 --2- 172.21.15.33:0/2024077033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c003a90 0x7f104014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.913+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 >> 172.21.15.33:0/2024077033 conn(0x7f10400c97b0 msgr2=0x7f104005db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.913+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 shutdown_connections 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:19.913+0000 7f1045dfa700 1 -- 172.21.15.33:0/2024077033 wait complete. 2023-10-06T02:45:19.918 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2023-10-06T02:45:19.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:20.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 Processor -- start 2023-10-06T02:45:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 -- start start 2023-10-06T02:45:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 0x7f841c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f841c158fc0 0x7f841c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c15d830 con 0x7f841c0b0530 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c067c10 con 0x7f841c158fc0 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f8420898700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0b00a0 con 0x7f841c158bf0 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 0x7f841c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 0x7f841c0b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40322/0 (socket says 172.21.15.33:40322) 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 -- 172.21.15.33:0/1485983894 learned_addr learned my addr 172.21.15.33:0/1485983894 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841ad9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f841c158fc0 0x7f841c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 -- 172.21.15.33:0/1485983894 >> v2:172.21.15.167:3300/0 conn(0x7f841c158fc0 msgr2=0x7f841c15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 --2- 172.21.15.33:0/1485983894 >> v2:172.21.15.167:3300/0 conn(0x7f841c158fc0 0x7f841c15d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 -- 172.21.15.33:0/1485983894 >> v1:172.21.15.167:6789/0 conn(0x7f841c0b0530 legacy=0x7f841c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 -- 172.21.15.33:0/1485983894 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c0cba20 con 0x7f841c158bf0 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841959a700 1 -- 172.21.15.33:0/1485983894 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512163474 0 0) 0x7f841c15d830 con 0x7f841c0b0530 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.025+0000 7f841a59c700 1 --2- 172.21.15.33:0/1485983894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 0x7f841c0b1b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f840c009f20 tx=0x7f840c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7f66fca289baab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f841959a700 1 -- 172.21.15.33:0/1485983894 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00e070 con 0x7f841c158bf0 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f841959a700 1 -- 172.21.15.33:0/1485983894 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00b9d0 con 0x7f841c158bf0 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f841959a700 1 -- 172.21.15.33:0/1485983894 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c0092e0 con 0x7f841c158bf0 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 -- 172.21.15.33:0/1485983894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 msgr2=0x7f841c0b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 --2- 172.21.15.33:0/1485983894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 0x7f841c0b1b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f840c009f20 tx=0x7f840c009840 comp rx=0 tx=0).stop 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 -- 172.21.15.33:0/1485983894 shutdown_connections 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 --2- 172.21.15.33:0/1485983894 >> v2:172.21.15.167:3300/0 conn(0x7f841c158fc0 0x7f841c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 --2- 172.21.15.33:0/1485983894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c158bf0 0x7f841c0b1b90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 -- 172.21.15.33:0/1485983894 >> 172.21.15.33:0/1485983894 conn(0x7f841c0cfd70 msgr2=0x7f841c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 -- 172.21.15.33:0/1485983894 shutdown_connections 2023-10-06T02:45:20.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 -- 172.21.15.33:0/1485983894 wait complete. 2023-10-06T02:45:20.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 Processor -- start 2023-10-06T02:45:20.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.029+0000 7f8420898700 1 -- start start 2023-10-06T02:45:20.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 0x7f841c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c158fc0 0x7f841c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c150160 0x7f841c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0c61b0 con 0x7f841c158bf0 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c159af0 con 0x7f841c158fc0 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c159dd0 con 0x7f841c150160 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 0x7f841c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 0x7f841c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54424/0 (socket says 172.21.15.33:54424) 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 -- 172.21.15.33:0/1055036079 learned_addr learned my addr 172.21.15.33:0/1055036079 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841ad9d700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c150160 0x7f841c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8419d9b700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c158fc0 0x7f841c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 -- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c150160 msgr2=0x7f841c068380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c150160 0x7f841c068380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 -- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c158fc0 msgr2=0x7f841c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c158fc0 0x7f841c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 -- 172.21.15.33:0/1055036079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009ae0 con 0x7f841c158bf0 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8419d9b700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c158fc0 0x7f841c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f841a59c700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 0x7f841c159460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f841c0b01e0 tx=0x7f840c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c008920 con 0x7f841c158bf0 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c008a80 con 0x7f841c158bf0 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c15a0b0 con 0x7f841c158bf0 2023-10-06T02:45:20.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c008e40 con 0x7f841c158bf0 2023-10-06T02:45:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c15a670 con 0x7f841c158bf0 2023-10-06T02:45:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.033+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c15a890 con 0x7f841c158bf0 2023-10-06T02:45:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.037+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f840c003040 con 0x7f841c158bf0 2023-10-06T02:45:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.037+0000 7f840a7fc700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83fc060550 0x7f83fc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.037+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f840c07e3f0 con 0x7f841c158bf0 2023-10-06T02:45:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.037+0000 7f8419d9b700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83fc060550 0x7f83fc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.037+0000 7f8419d9b700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83fc060550 0x7f83fc062a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84100011e0 tx=0x7f8410009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.045+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f840c04ec80 con 0x7f841c158bf0 2023-10-06T02:45:20.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.209+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f841c0710b0 con 0x7f841c158bf0 2023-10-06T02:45:20.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f840a7fc700 1 -- 172.21.15.33:0/1055036079 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+13647 (secure 0 0 0) 0x7f840c0522a0 con 0x7f841c158bf0 2023-10-06T02:45:20.218 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:20.219 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:19.630548+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83fc060550 msgr2=0x7f83fc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83fc060550 0x7f83fc062a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84100011e0 tx=0x7f8410009040 comp rx=0 tx=0).stop 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 msgr2=0x7f841c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 0x7f841c159460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f841c0b01e0 tx=0x7f840c018000 comp rx=0 tx=0).stop 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 shutdown_connections 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83fc060550 0x7f83fc062a00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f841c150160 0x7f841c068380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841c158fc0 0x7f841c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 --2- 172.21.15.33:0/1055036079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f841c158bf0 0x7f841c159460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.213+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 >> 172.21.15.33:0/1055036079 conn(0x7f841c0cfd70 msgr2=0x7f841c0aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.217+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 shutdown_connections 2023-10-06T02:45:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.217+0000 7f8420898700 1 -- 172.21.15.33:0/1055036079 wait complete. 2023-10-06T02:45:20.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:45:20.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:45:20.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 Processor -- start 2023-10-06T02:45:20.346 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 -- start start 2023-10-06T02:45:20.347 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0b900abf30 0x7f0b900adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b900ae500 0x7f0b900678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b900af1e0 con 0x7f0b90156bd0 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b900b0890 con 0x7f0b900ae500 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b96938700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b90067450 con 0x7f0b900abf30 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b900ae500 0x7f0b900678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b900ae500 0x7f0b900678e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54440/0 (socket says 172.21.15.33:54440) 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 -- 172.21.15.33:0/1925238558 learned_addr learned my addr 172.21.15.33:0/1925238558 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b877fe700 1 --2- 172.21.15.33:0/1925238558 >> v2:172.21.15.167:3300/0 conn(0x7f0b900abf30 0x7f0b900adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 -- 172.21.15.33:0/1925238558 >> v2:172.21.15.167:3300/0 conn(0x7f0b900abf30 msgr2=0x7f0b900adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 --2- 172.21.15.33:0/1925238558 >> v2:172.21.15.167:3300/0 conn(0x7f0b900abf30 0x7f0b900adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 -- 172.21.15.33:0/1925238558 >> v1:172.21.15.167:6789/0 conn(0x7f0b90156bd0 legacy=0x7f0b900ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 -- 172.21.15.33:0/1925238558 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900aec20 con 0x7f0b900ae500 2023-10-06T02:45:20.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b94ed5700 1 --2- 172.21.15.33:0/1925238558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b900678e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b9006d8f0 tx=0x7f0b88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ced9a8072bec9f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b8f7fe700 1 -- 172.21.15.33:0/1925238558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800e070 con 0x7f0b900ae500 2023-10-06T02:45:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b8f7fe700 1 -- 172.21.15.33:0/1925238558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8800ba10 con 0x7f0b900ae500 2023-10-06T02:45:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.341+0000 7f0b8f7fe700 1 -- 172.21.15.33:0/1925238558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b880092e0 con 0x7f0b900ae500 2023-10-06T02:45:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/1925238558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 msgr2=0x7f0b900678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 --2- 172.21.15.33:0/1925238558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b900678e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b9006d8f0 tx=0x7f0b88009840 comp rx=0 tx=0).stop 2023-10-06T02:45:20.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/1925238558 shutdown_connections 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 --2- 172.21.15.33:0/1925238558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b900678e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 --2- 172.21.15.33:0/1925238558 >> v2:172.21.15.167:3300/0 conn(0x7f0b900abf30 0x7f0b900adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/1925238558 >> 172.21.15.33:0/1925238558 conn(0x7f0b900af9c0 msgr2=0x7f0b900afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/1925238558 shutdown_connections 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/1925238558 wait complete. 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 Processor -- start 2023-10-06T02:45:20.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- start start 2023-10-06T02:45:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900abf30 0x7f0b900cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b901696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b90156bd0 0x7f0b901485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90169bf0 con 0x7f0b900ae500 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90148af0 con 0x7f0b900abf30 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900cb790 con 0x7f0b90156bd0 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b901696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b901696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54442/0 (socket says 172.21.15.33:54442) 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 -- 172.21.15.33:0/733186695 learned_addr learned my addr 172.21.15.33:0/733186695 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900abf30 0x7f0b900cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b94ed5700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b90156bd0 0x7f0b901485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 -- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b90156bd0 msgr2=0x7f0b901485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b90156bd0 0x7f0b901485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 -- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900abf30 msgr2=0x7f0b900cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900abf30 0x7f0b900cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 -- 172.21.15.33:0/733186695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88009ae0 con 0x7f0b900ae500 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b877fe700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b901696b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d890 tx=0x7f0b78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780125f0 con 0x7f0b900ae500 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b78012750 con 0x7f0b900ae500 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b900cba70 con 0x7f0b900ae500 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78012ad0 con 0x7f0b900ae500 2023-10-06T02:45:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.345+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b900a98f0 con 0x7f0b900ae500 2023-10-06T02:45:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.349+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b900710b0 con 0x7f0b900ae500 2023-10-06T02:45:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.349+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0b7801e460 con 0x7f0b900ae500 2023-10-06T02:45:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.349+0000 7f0b8cff9700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0b74060550 0x7f0b74062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.349+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0b7807cc80 con 0x7f0b900ae500 2023-10-06T02:45:20.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.349+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0b74060550 0x7f0b74062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.353+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0b74060550 0x7f0b74062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b9015c4a0 tx=0x7f0b80009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.357+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0b7804d460 con 0x7f0b900ae500 2023-10-06T02:45:20.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b9006a160 con 0x7f0b900ae500 2023-10-06T02:45:20.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b8cff9700 1 -- 172.21.15.33:0/733186695 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b780530a0 con 0x7f0b900ae500 2023-10-06T02:45:20.523 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0b74060550 msgr2=0x7f0b74062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0b74060550 0x7f0b74062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b9015c4a0 tx=0x7f0b80009060 comp rx=0 tx=0).stop 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 msgr2=0x7f0b901696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b901696b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d890 tx=0x7f0b78008040 comp rx=0 tx=0).stop 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 shutdown_connections 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0b74060550 0x7f0b74062a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b90156bd0 0x7f0b901485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900ae500 0x7f0b901696b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 --2- 172.21.15.33:0/733186695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b900abf30 0x7f0b900cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.517+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 >> 172.21.15.33:0/733186695 conn(0x7f0b900af9c0 msgr2=0x7f0b9014a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.521+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 shutdown_connections 2023-10-06T02:45:20.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.521+0000 7f0b96938700 1 -- 172.21.15.33:0/733186695 wait complete. 2023-10-06T02:45:20.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:45:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 Processor -- start 2023-10-06T02:45:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 -- start start 2023-10-06T02:45:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3bb815a1e0 0x7f3bb80ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb80cc900 0x7f3bb80cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb80c9d80 con 0x7f3bb80c3480 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb80ca060 con 0x7f3bb815a1e0 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bc0541700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c29a0 con 0x7f3bb80cc900 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bbeade700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3bb80c3480 0x7f3bb80c38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33484/0 (socket says 172.21.15.33:33484) 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bbeade700 1 -- 172.21.15.33:0/3062412467 learned_addr learned my addr 172.21.15.33:0/3062412467 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.641+0000 7f3bbe2dd700 1 --2- 172.21.15.33:0/3062412467 >> v2:172.21.15.167:3300/0 conn(0x7f3bb815a1e0 0x7f3bb80ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.645+0000 7f3bbdadc700 1 --2- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb80cc900 0x7f3bb80cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.645+0000 7f3bbe2dd700 1 -- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb80cc900 msgr2=0x7f3bb80cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.645+0000 7f3bbe2dd700 1 --2- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb80cc900 0x7f3bb80cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.645+0000 7f3bbe2dd700 1 -- 172.21.15.33:0/3062412467 >> v1:172.21.15.33:6789/0 conn(0x7f3bb80c3480 legacy=0x7f3bb80c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.645+0000 7f3bbe2dd700 1 -- 172.21.15.33:0/3062412467 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb80c2c80 con 0x7f3bb815a1e0 2023-10-06T02:45:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.645+0000 7f3bbe2dd700 1 --2- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb815a1e0 0x7f3bb80ca5d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098e0 tx=0x7f3bb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=decac2b8284427da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bbd2db700 1 -- 172.21.15.33:0/3062412467 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400b640 con 0x7f3bb815a1e0 2023-10-06T02:45:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bbd2db700 1 -- 172.21.15.33:0/3062412467 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb400b7a0 con 0x7f3bb815a1e0 2023-10-06T02:45:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bbd2db700 1 -- 172.21.15.33:0/3062412467 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400bb40 con 0x7f3bb815a1e0 2023-10-06T02:45:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 -- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb815a1e0 msgr2=0x7f3bb80ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 --2- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb815a1e0 0x7f3bb80ca5d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098e0 tx=0x7f3bb40091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 -- 172.21.15.33:0/3062412467 shutdown_connections 2023-10-06T02:45:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 --2- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb80cc900 0x7f3bb80cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 --2- 172.21.15.33:0/3062412467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb815a1e0 0x7f3bb80ca5d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 -- 172.21.15.33:0/3062412467 >> 172.21.15.33:0/3062412467 conn(0x7f3bb8096ef0 msgr2=0x7f3bb806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 -- 172.21.15.33:0/3062412467 shutdown_connections 2023-10-06T02:45:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 -- 172.21.15.33:0/3062412467 wait complete. 2023-10-06T02:45:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 Processor -- start 2023-10-06T02:45:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.649+0000 7f3bc0541700 1 -- start start 2023-10-06T02:45:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb80c3480 0x7f3bb80c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 0x7f3bb8154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb815a1e0 0x7f3bb8140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c8110 con 0x7f3bb80cc900 2023-10-06T02:45:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8153b50 con 0x7f3bb80c3480 2023-10-06T02:45:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c7590 con 0x7f3bb815a1e0 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 0x7f3bb8154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 0x7f3bb8154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54446/0 (socket says 172.21.15.33:54446) 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 -- 172.21.15.33:0/1095137252 learned_addr learned my addr 172.21.15.33:0/1095137252 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbeade700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb815a1e0 0x7f3bb8140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbe2dd700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb80c3480 0x7f3bb80c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 -- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb815a1e0 msgr2=0x7f3bb8140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb815a1e0 0x7f3bb8140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 -- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb80c3480 msgr2=0x7f3bb80c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb80c3480 0x7f3bb80c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 -- 172.21.15.33:0/1095137252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4009420 con 0x7f3bb80cc900 2023-10-06T02:45:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bbdadc700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 0x7f3bb8154080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800ec50 tx=0x7f3ba80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800c850 con 0x7f3bb80cc900 2023-10-06T02:45:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba800c9b0 con 0x7f3bb80cc900 2023-10-06T02:45:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb80c7870 con 0x7f3bb80cc900 2023-10-06T02:45:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800cd70 con 0x7f3bb80cc900 2023-10-06T02:45:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.653+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb80c7e30 con 0x7f3bb80cc900 2023-10-06T02:45:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.657+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3ba8013680 con 0x7f3bb80cc900 2023-10-06T02:45:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.657+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb80710b0 con 0x7f3bb80cc900 2023-10-06T02:45:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.657+0000 7f3bae7fc700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3b9c060550 0x7f3b9c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.657+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3ba807cb70 con 0x7f3bb80cc900 2023-10-06T02:45:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.657+0000 7f3bbe2dd700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3b9c060550 0x7f3b9c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.661+0000 7f3bbe2dd700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3b9c060550 0x7f3b9c062a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40093f0 tx=0x7f3bb4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.672 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.665+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3ba804d350 con 0x7f3bb80cc900 2023-10-06T02:45:20.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.825+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3bb80b09a0 con 0x7f3bb80cc900 2023-10-06T02:45:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bae7fc700 1 -- 172.21.15.33:0/1095137252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ba8050970 con 0x7f3bb80cc900 2023-10-06T02:45:20.833 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3b9c060550 msgr2=0x7f3b9c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3b9c060550 0x7f3b9c062a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40093f0 tx=0x7f3bb4017000 comp rx=0 tx=0).stop 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 msgr2=0x7f3bb8154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 0x7f3bb8154080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800ec50 tx=0x7f3ba80091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 shutdown_connections 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3b9c060550 0x7f3b9c062a00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb815a1e0 0x7f3bb8140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb80cc900 0x7f3bb8154080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 --2- 172.21.15.33:0/1095137252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb80c3480 0x7f3bb80c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 >> 172.21.15.33:0/1095137252 conn(0x7f3bb8096ef0 msgr2=0x7f3bb80c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 shutdown_connections 2023-10-06T02:45:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.829+0000 7f3bc0541700 1 -- 172.21.15.33:0/1095137252 wait complete. 2023-10-06T02:45:20.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:45:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4a12c700 1 Processor -- start 2023-10-06T02:45:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4a12c700 1 -- start start 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4a12c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5f441428b0 0x7f5f44068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4a12c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440cf3e0 con 0x7f5f440688d0 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4a12c700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440ce7d0 con 0x7f5f441424e0 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4a12c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f44068d70 con 0x7f5f441428b0 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4992b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5f440688d0 0x7f5f440b29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33490/0 (socket says 172.21.15.33:33490) 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f4992b700 1 -- 172.21.15.33:0/2850879876 learned_addr learned my addr 172.21.15.33:0/2850879876 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f48929700 1 --2- 172.21.15.33:0/2850879876 >> v2:172.21.15.167:3300/0 conn(0x7f5f441428b0 0x7f5f44068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207882120 0 0) 0x7f5f440cf3e0 con 0x7f5f440688d0 2023-10-06T02:45:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f30003570 con 0x7f5f440688d0 2023-10-06T02:45:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344095343 0 0) 0x7f5f30003570 con 0x7f5f440688d0 2023-10-06T02:45:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 >> v2:172.21.15.167:3300/0 conn(0x7f5f441428b0 msgr2=0x7f5f44068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 --2- 172.21.15.33:0/2850879876 >> v2:172.21.15.167:3300/0 conn(0x7f5f441428b0 0x7f5f44068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 >> v1:172.21.15.167:6789/0 conn(0x7f5f441424e0 legacy=0x7f5f440cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440ceab0 con 0x7f5f440688d0 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f5f3c0017f0 con 0x7f5f440688d0 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.953+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 >> v1:172.21.15.33:6789/0 conn(0x7f5f440688d0 legacy=0x7f5f440b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f30003a60 0x7f5f30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f30004810 0x7f5f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f300051c0 0x7f5f3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44068d70 con 0x7f5f300051c0 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3000c0d0 con 0x7f5f30003a60 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3000c290 con 0x7f5f30004810 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f300051c0 0x7f5f3000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f48929700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f30004810 0x7f5f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4912a700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f30003a60 0x7f5f30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 -- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f30004810 msgr2=0x7f5f30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f30004810 0x7f5f30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 -- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f30003a60 msgr2=0x7f5f30003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f30003a60 0x7f5f30003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 -- 172.21.15.33:0/2850879876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30003570 con 0x7f5f300051c0 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f48929700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f30004810 0x7f5f30004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 -- 172.21.15.33:0/2850879876 reap_dead start 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4992b700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f300051c0 0x7f5f3000ba60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00a930 tx=0x7f5f3c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fc8e4dd5d6298d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0070d0 con 0x7f5f300051c0 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c007230 con 0x7f5f300051c0 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f3bfff700 1 -- 172.21.15.33:0/2850879876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0075d0 con 0x7f5f300051c0 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 -- 172.21.15.33:0/2850879876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f300051c0 msgr2=0x7f5f3000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f300051c0 0x7f5f3000ba60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00a930 tx=0x7f5f3c0021d0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 -- 172.21.15.33:0/2850879876 shutdown_connections 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/2850879876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f300051c0 0x7f5f3000ba60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 -- 172.21.15.33:0/2850879876 >> 172.21.15.33:0/2850879876 conn(0x7f5f44067a40 msgr2=0x7f5f440b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 -- 172.21.15.33:0/2850879876 shutdown_connections 2023-10-06T02:45:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.957+0000 7f5f4a12c700 1 -- 172.21.15.33:0/2850879876 wait complete. 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 Processor -- start 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- start start 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f300051c0 0x7f5f441587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440b0760 0x7f5f440b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 0x7f5f440c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440c5a30 con 0x7f5f440c4cc0 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b1110 con 0x7f5f300051c0 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44168ff0 con 0x7f5f440b0760 2023-10-06T02:45:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 0x7f5f440c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 0x7f5f440c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54470/0 (socket says 172.21.15.33:54470) 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 -- 172.21.15.33:0/1526183231 learned_addr learned my addr 172.21.15.33:0/1526183231 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f48929700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440b0760 0x7f5f440b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4912a700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f300051c0 0x7f5f441587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 -- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440b0760 msgr2=0x7f5f440b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440b0760 0x7f5f440b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 -- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f300051c0 msgr2=0x7f5f441587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f300051c0 0x7f5f441587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 -- 172.21.15.33:0/1526183231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440c7010 con 0x7f5f440c4cc0 2023-10-06T02:45:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4912a700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f300051c0 0x7f5f441587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4992b700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 0x7f5f440c5110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000f80 tx=0x7f5f3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c022070 con 0x7f5f440c4cc0 2023-10-06T02:45:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c007b50 con 0x7f5f440c4cc0 2023-10-06T02:45:20.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f440c66e0 con 0x7f5f440c4cc0 2023-10-06T02:45:20.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0070d0 con 0x7f5f440c4cc0 2023-10-06T02:45:20.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f440c6ca0 con 0x7f5f440c4cc0 2023-10-06T02:45:20.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.961+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f440c71a0 con 0x7f5f440c4cc0 2023-10-06T02:45:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.965+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5f3c031430 con 0x7f5f440c4cc0 2023-10-06T02:45:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.965+0000 7f5f397fa700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5f28060550 0x7f5f28062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.965+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5f3c07ed90 con 0x7f5f440c4cc0 2023-10-06T02:45:20.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.965+0000 7f5f4912a700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5f28060550 0x7f5f28062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:20.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.965+0000 7f5f4912a700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5f28060550 0x7f5f28062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f40000c00 tx=0x7f5f40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:20.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:20.973+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5f3c04f570 con 0x7f5f440c4cc0 2023-10-06T02:45:21.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.133+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f44071040 con 0x7f5f440c4cc0 2023-10-06T02:45:21.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.133+0000 7f5f397fa700 1 -- 172.21.15.33:0/1526183231 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v21) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5f3c0017b0 con 0x7f5f440c4cc0 2023-10-06T02:45:21.140 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:21.140 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T02:45:21.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:45:21.140 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T02:45:21.140 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T02:45:21.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:21.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:45:21.141 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T02:45:21.141 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T02:45:21.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:45:21.141 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5f28060550 msgr2=0x7f5f28062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5f28060550 0x7f5f28062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f40000c00 tx=0x7f5f40009040 comp rx=0 tx=0).stop 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 msgr2=0x7f5f440c5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 0x7f5f440c5110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000f80 tx=0x7f5f3c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 shutdown_connections 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5f28060550 0x7f5f28062a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f440b0760 0x7f5f440b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f440c4cc0 0x7f5f440c5110 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 --2- 172.21.15.33:0/1526183231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f300051c0 0x7f5f441587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 >> 172.21.15.33:0/1526183231 conn(0x7f5f44067a40 msgr2=0x7f5f440ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:21.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 shutdown_connections 2023-10-06T02:45:21.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.137+0000 7f5f4a12c700 1 -- 172.21.15.33:0/1526183231 wait complete. 2023-10-06T02:45:21.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:45:21.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:45:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 Processor -- start 2023-10-06T02:45:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 -- start start 2023-10-06T02:45:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01680c6650 0x7f01680ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0168160cf0 0x7f0168161170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0168067710 con 0x7f01680beb30 2023-10-06T02:45:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01680aecd0 con 0x7f01680c6650 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016e303700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01681616b0 con 0x7f0168160cf0 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016cb00700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f01680c6650 0x7f01680ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016d301700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f01680beb30 0x7f01680b1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33500/0 (socket says 172.21.15.33:33500) 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016d301700 1 -- 172.21.15.33:0/885471155 learned_addr learned my addr 172.21.15.33:0/885471155 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016db02700 1 --2- 172.21.15.33:0/885471155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0168160cf0 0x7f0168161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f015ffff700 1 -- 172.21.15.33:0/885471155 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323065895 0 0) 0x7f0168067710 con 0x7f01680beb30 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f015ffff700 1 -- 172.21.15.33:0/885471155 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0154003570 con 0x7f01680beb30 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016cb00700 1 -- 172.21.15.33:0/885471155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0168160cf0 msgr2=0x7f0168161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016cb00700 1 --2- 172.21.15.33:0/885471155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0168160cf0 0x7f0168161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016cb00700 1 -- 172.21.15.33:0/885471155 >> v1:172.21.15.33:6789/0 conn(0x7f01680beb30 legacy=0x7f01680b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016cb00700 1 -- 172.21.15.33:0/885471155 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168161b50 con 0x7f01680c6650 2023-10-06T02:45:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f016cb00700 1 --2- 172.21.15.33:0/885471155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01680c6650 0x7f01680ae790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0158009910 tx=0x7f01580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917f199418d68191 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f015ffff700 1 -- 172.21.15.33:0/885471155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158011630 con 0x7f01680c6650 2023-10-06T02:45:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f015ffff700 1 -- 172.21.15.33:0/885471155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0158011790 con 0x7f01680c6650 2023-10-06T02:45:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.265+0000 7f015ffff700 1 -- 172.21.15.33:0/885471155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158011b10 con 0x7f01680c6650 2023-10-06T02:45:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- 172.21.15.33:0/885471155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01680c6650 msgr2=0x7f01680ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 --2- 172.21.15.33:0/885471155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01680c6650 0x7f01680ae790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0158009910 tx=0x7f01580091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- 172.21.15.33:0/885471155 shutdown_connections 2023-10-06T02:45:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 --2- 172.21.15.33:0/885471155 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0168160cf0 0x7f0168161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 --2- 172.21.15.33:0/885471155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01680c6650 0x7f01680ae790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- 172.21.15.33:0/885471155 >> 172.21.15.33:0/885471155 conn(0x7f01680cc290 msgr2=0x7f01680c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- 172.21.15.33:0/885471155 shutdown_connections 2023-10-06T02:45:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- 172.21.15.33:0/885471155 wait complete. 2023-10-06T02:45:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 Processor -- start 2023-10-06T02:45:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- start start 2023-10-06T02:45:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01680beb30 0x7f016815c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01680c6650 0x7f01680bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 0x7f01680c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016815d7a0 con 0x7f0168160cf0 2023-10-06T02:45:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01680c3340 con 0x7f01680c6650 2023-10-06T02:45:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01680c3620 con 0x7f01680beb30 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 0x7f01680c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 0x7f01680c3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54496/0 (socket says 172.21.15.33:54496) 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 -- 172.21.15.33:0/3891519512 learned_addr learned my addr 172.21.15.33:0/3891519512 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016cb00700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01680c6650 0x7f01680bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016d301700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01680beb30 0x7f016815c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 -- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01680beb30 msgr2=0x7f016815c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01680beb30 0x7f016815c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 -- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01680c6650 msgr2=0x7f01680bdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01680c6650 0x7f01680bdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 -- 172.21.15.33:0/3891519512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0158009420 con 0x7f0168160cf0 2023-10-06T02:45:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016db02700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 0x7f01680c3cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0160009df0 tx=0x7f016000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000caa0 con 0x7f0168160cf0 2023-10-06T02:45:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016000f040 con 0x7f0168160cf0 2023-10-06T02:45:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.269+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01680c3900 con 0x7f0168160cf0 2023-10-06T02:45:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000ce70 con 0x7f0168160cf0 2023-10-06T02:45:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016815fcd0 con 0x7f0168160cf0 2023-10-06T02:45:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016815d910 con 0x7f0168160cf0 2023-10-06T02:45:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0160014690 con 0x7f0168160cf0 2023-10-06T02:45:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f015d7fa700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f014c060550 0x7f014c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f016007cf20 con 0x7f0168160cf0 2023-10-06T02:45:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.273+0000 7f016d301700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f014c060550 0x7f014c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.277+0000 7f016d301700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f014c060550 0x7f014c062a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0164007fd0 tx=0x7f0164009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:21.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.285+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f01600510f0 con 0x7f0168160cf0 2023-10-06T02:45:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.441+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0168071040 con 0x7f0168160cf0 2023-10-06T02:45:21.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.765+0000 7f015d7fa700 1 -- 172.21.15.33:0/3891519512 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v22) v1 ==== 138+0+0 (secure 0 0 0) 0x7f01600180b0 con 0x7f0168160cf0 2023-10-06T02:45:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f014c060550 msgr2=0x7f014c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f014c060550 0x7f014c062a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0164007fd0 tx=0x7f0164009040 comp rx=0 tx=0).stop 2023-10-06T02:45:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 msgr2=0x7f01680c3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 0x7f01680c3cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0160009df0 tx=0x7f016000b040 comp rx=0 tx=0).stop 2023-10-06T02:45:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 shutdown_connections 2023-10-06T02:45:21.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168160cf0 0x7f01680c3cc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01680c6650 0x7f01680bdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f014c060550 0x7f014c062a00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 --2- 172.21.15.33:0/3891519512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01680beb30 0x7f016815c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 >> 172.21.15.33:0/3891519512 conn(0x7f01680cc290 msgr2=0x7f01680ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 shutdown_connections 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.777+0000 7f016e303700 1 -- 172.21.15.33:0/3891519512 wait complete. 2023-10-06T02:45:21.785 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T02:45:21.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-06T02:45:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.913+0000 7ff545d45700 1 Processor -- start 2023-10-06T02:45:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.913+0000 7ff545d45700 1 -- start start 2023-10-06T02:45:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff540158ca0 0x7ff5400cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff540159070 0x7ff5400b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400cb2f0 con 0x7ff5400b2f20 2023-10-06T02:45:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54015d9c0 con 0x7ff540159070 2023-10-06T02:45:21.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400caa00 con 0x7ff540158ca0 2023-10-06T02:45:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff540159070 0x7ff5400b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545544700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff5400b2f20 0x7ff540066fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33516/0 (socket says 172.21.15.33:33516) 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545544700 1 -- 172.21.15.33:0/2921591284 learned_addr learned my addr 172.21.15.33:0/2921591284 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff544d43700 1 --2- 172.21.15.33:0/2921591284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff540158ca0 0x7ff5400cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53ffff700 1 -- 172.21.15.33:0/2921591284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff540158ca0 msgr2=0x7ff5400cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53ffff700 1 --2- 172.21.15.33:0/2921591284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff540158ca0 0x7ff5400cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53ffff700 1 -- 172.21.15.33:0/2921591284 >> v1:172.21.15.33:6789/0 conn(0x7ff5400b2f20 legacy=0x7ff540066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53ffff700 1 -- 172.21.15.33:0/2921591284 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400b33d0 con 0x7ff540159070 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53ffff700 1 --2- 172.21.15.33:0/2921591284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540159070 0x7ff5400b29e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098e0 tx=0x7ff5300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e012f26a778a089e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53f7fe700 1 -- 172.21.15.33:0/2921591284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000b640 con 0x7ff540159070 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53f7fe700 1 -- 172.21.15.33:0/2921591284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53000b7a0 con 0x7ff540159070 2023-10-06T02:45:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff53f7fe700 1 -- 172.21.15.33:0/2921591284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000bb40 con 0x7ff540159070 2023-10-06T02:45:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- 172.21.15.33:0/2921591284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540159070 msgr2=0x7ff5400b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 --2- 172.21.15.33:0/2921591284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540159070 0x7ff5400b29e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098e0 tx=0x7ff5300091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- 172.21.15.33:0/2921591284 shutdown_connections 2023-10-06T02:45:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 --2- 172.21.15.33:0/2921591284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540159070 0x7ff5400b29e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 --2- 172.21.15.33:0/2921591284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff540158ca0 0x7ff5400cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- 172.21.15.33:0/2921591284 >> 172.21.15.33:0/2921591284 conn(0x7ff5400cc680 msgr2=0x7ff5400cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- 172.21.15.33:0/2921591284 shutdown_connections 2023-10-06T02:45:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.917+0000 7ff545d45700 1 -- 172.21.15.33:0/2921591284 wait complete. 2023-10-06T02:45:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 Processor -- start 2023-10-06T02:45:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 -- start start 2023-10-06T02:45:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400b2f20 0x7ff540155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 0x7ff5400c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff540159070 0x7ff5400cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5401559b0 con 0x7ff540158ca0 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400c76e0 con 0x7ff540159070 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400c79c0 con 0x7ff5400b2f20 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 0x7ff5400c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 0x7ff5400c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54502/0 (socket says 172.21.15.33:54502) 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 -- 172.21.15.33:0/386082289 learned_addr learned my addr 172.21.15.33:0/386082289 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff544d43700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400b2f20 0x7ff540155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545544700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff540159070 0x7ff5400cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 -- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400b2f20 msgr2=0x7ff540155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400b2f20 0x7ff540155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 -- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff540159070 msgr2=0x7ff5400cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff540159070 0x7ff5400cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 -- 172.21.15.33:0/386082289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530009420 con 0x7ff540158ca0 2023-10-06T02:45:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53ffff700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 0x7ff5400c6c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff530009f80 tx=0x7ff530017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530000d10 con 0x7ff540158ca0 2023-10-06T02:45:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53000bb20 con 0x7ff540158ca0 2023-10-06T02:45:21.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530011670 con 0x7ff540158ca0 2023-10-06T02:45:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5400c7ca0 con 0x7ff540158ca0 2023-10-06T02:45:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.921+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5401694c0 con 0x7ff540158ca0 2023-10-06T02:45:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.925+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5401696e0 con 0x7ff540158ca0 2023-10-06T02:45:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.925+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff530011aa0 con 0x7ff540158ca0 2023-10-06T02:45:21.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.925+0000 7ff53cff9700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff520060550 0x7ff520062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.925+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5576+0+0 (secure 0 0 0) 0x7ff53007e6a0 con 0x7ff540158ca0 2023-10-06T02:45:21.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.925+0000 7ff544d43700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff520060550 0x7ff520062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.929+0000 7ff544d43700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff520060550 0x7ff520062a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff528009db0 tx=0x7ff5280091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:21.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:21.933+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff53004ed90 con 0x7ff540158ca0 2023-10-06T02:45:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.097+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7ff540071040 con 0x7ff540158ca0 2023-10-06T02:45:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.781+0000 7ff53cff9700 1 -- 172.21.15.33:0/386082289 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v23) v1 ==== 171+0+0 (secure 0 0 0) 0x7ff5300523b0 con 0x7ff540158ca0 2023-10-06T02:45:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff520060550 msgr2=0x7ff520062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff520060550 0x7ff520062a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff528009db0 tx=0x7ff5280091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 msgr2=0x7ff5400c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 0x7ff5400c6c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff530009f80 tx=0x7ff530017000 comp rx=0 tx=0).stop 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 shutdown_connections 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff540159070 0x7ff5400cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff540158ca0 0x7ff5400c6c20 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff530009f80 tx=0x7ff530017000 comp rx=0 tx=0).stop 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff520060550 0x7ff520062a00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 --2- 172.21.15.33:0/386082289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5400b2f20 0x7ff540155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.793+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 >> 172.21.15.33:0/386082289 conn(0x7ff5400cc680 msgr2=0x7ff5400cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.801+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 shutdown_connections 2023-10-06T02:45:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.801+0000 7ff545d45700 1 -- 172.21.15.33:0/386082289 wait complete. 2023-10-06T02:45:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T02:45:22.822 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:45:22.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.921+0000 7f7d90d0e700 1 Processor -- start 2023-10-06T02:45:22.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d90d0e700 1 -- start start 2023-10-06T02:45:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d90d0e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7d8c0b0530 0x7f7d8c15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d90d0e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8c15d830 con 0x7f7d8c158fc0 2023-10-06T02:45:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d90d0e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8c0b00a0 con 0x7f7d8c158bf0 2023-10-06T02:45:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d90d0e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8c067c10 con 0x7f7d8c0b0530 2023-10-06T02:45:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8ad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7d8c158fc0 0x7f7d8c15cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60296/0 (socket says 172.21.15.33:60296) 2023-10-06T02:45:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8ad9d700 1 -- 172.21.15.33:0/2928485473 learned_addr learned my addr 172.21.15.33:0/2928485473 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/2928485473 >> v2:172.21.15.167:3300/0 conn(0x7f7d8c0b0530 0x7f7d8c15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893343999 0 0) 0x7f7d8c15d830 con 0x7f7d8c158fc0 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d74003570 con 0x7f7d8c158fc0 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2395043322 0 0) 0x7f7d74003570 con 0x7f7d8c158fc0 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 >> v2:172.21.15.167:3300/0 conn(0x7f7d8c0b0530 msgr2=0x7f7d8c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 --2- 172.21.15.33:0/2928485473 >> v2:172.21.15.167:3300/0 conn(0x7f7d8c0b0530 0x7f7d8c15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 >> v1:172.21.15.167:6789/0 conn(0x7f7d8c158bf0 legacy=0x7f7d8c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c0cba20 con 0x7f7d8c158fc0 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f7d840017f0 con 0x7f7d8c158fc0 2023-10-06T02:45:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/2928485473 >> v2:172.21.15.167:3300/0 conn(0x7f7d8c0b0530 0x7f7d8c15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 >> v1:172.21.15.33:6789/0 conn(0x7f7d8c158fc0 legacy=0x7f7d8c15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d74003a60 0x7f7d74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d740051c0 0x7f7d7400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400c130 con 0x7f7d74004810 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400c2a0 con 0x7f7d74003a60 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400c460 con 0x7f7d740051c0 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8a59c700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d74003a60 0x7f7d74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8ad9d700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d740051c0 0x7f7d7400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 -- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d740051c0 msgr2=0x7f7d7400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:22.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d740051c0 0x7f7d7400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 -- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d74003a60 msgr2=0x7f7d74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:22.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d74003a60 0x7f7d74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 -- 172.21.15.33:0/2928485473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74003570 con 0x7f7d74004810 2023-10-06T02:45:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8ad9d700 1 -- 172.21.15.33:0/2928485473 reap_dead start 2023-10-06T02:45:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d74004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009df0 tx=0x7f7d800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe81dba004f03372 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.925+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80009740 con 0x7f7d74004810 2023-10-06T02:45:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d8959a700 1 -- 172.21.15.33:0/2928485473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000e920 con 0x7f7d74004810 2023-10-06T02:45:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 -- 172.21.15.33:0/2928485473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 msgr2=0x7f7d74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d74004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009df0 tx=0x7f7d800091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 -- 172.21.15.33:0/2928485473 shutdown_connections 2023-10-06T02:45:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/2928485473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d74004c80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 -- 172.21.15.33:0/2928485473 >> 172.21.15.33:0/2928485473 conn(0x7f7d8c0cfd70 msgr2=0x7f7d8c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 -- 172.21.15.33:0/2928485473 shutdown_connections 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 -- 172.21.15.33:0/2928485473 wait complete. 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 Processor -- start 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.929+0000 7f7d90d0e700 1 -- start start 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d8c154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d8c155290 0x7f7d8c1681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d8c1686e0 0x7f7d8c168b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c156120 con 0x7f7d74004810 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c159930 con 0x7f7d8c155290 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c155b60 con 0x7f7d8c1686e0 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d8c154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d8c154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36368/0 (socket says 172.21.15.33:36368) 2023-10-06T02:45:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 -- 172.21.15.33:0/1403548003 learned_addr learned my addr 172.21.15.33:0/1403548003 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d8c155290 0x7f7d8c1681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8ad9d700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d8c1686e0 0x7f7d8c168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 -- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d8c1686e0 msgr2=0x7f7d8c168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d8c1686e0 0x7f7d8c168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 -- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d8c155290 msgr2=0x7f7d8c1681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d8c155290 0x7f7d8c1681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 -- 172.21.15.33:0/1403548003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c155e40 con 0x7f7d74004810 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d8a59c700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d8c154d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e840 tx=0x7f7d7c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c012590 con 0x7f7d74004810 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7c0126f0 con 0x7f7d74004810 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8c1595c0 con 0x7f7d74004810 2023-10-06T02:45:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c012a90 con 0x7f7d74004810 2023-10-06T02:45:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8c155920 con 0x7f7d74004810 2023-10-06T02:45:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.933+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8c159420 con 0x7f7d74004810 2023-10-06T02:45:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.937+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7d7c01d460 con 0x7f7d74004810 2023-10-06T02:45:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.937+0000 7f7d7a7fc700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d6c060550 0x7f7d6c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.937+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f7d7c07c780 con 0x7f7d74004810 2023-10-06T02:45:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.937+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d6c060550 0x7f7d6c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.937+0000 7f7d89d9b700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d6c060550 0x7f7d6c062a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d800093f0 tx=0x7f7d8001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:22.945+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d8c159420 con 0x7f7d74004810 2023-10-06T02:45:23.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.141+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7d8c155920 con 0x7f7d74004810 2023-10-06T02:45:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.777+0000 7f7d7a7fc700 1 -- 172.21.15.33:0/1403548003 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7d8c155920 con 0x7f7d74004810 2023-10-06T02:45:23.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d6c060550 msgr2=0x7f7d6c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:23.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d6c060550 0x7f7d6c062a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d800093f0 tx=0x7f7d8001a000 comp rx=0 tx=0).stop 2023-10-06T02:45:23.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 msgr2=0x7f7d8c154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d8c154d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e840 tx=0x7f7d7c009040 comp rx=0 tx=0).stop 2023-10-06T02:45:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 shutdown_connections 2023-10-06T02:45:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d6c060550 0x7f7d6c062a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d8c1686e0 0x7f7d8c168b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d8c155290 0x7f7d8c1681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 --2- 172.21.15.33:0/1403548003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d74004810 0x7f7d8c154d50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 >> 172.21.15.33:0/1403548003 conn(0x7f7d8c0cfd70 msgr2=0x7f7d8c0cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.785+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 shutdown_connections 2023-10-06T02:45:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.789+0000 7f7d90d0e700 1 -- 172.21.15.33:0/1403548003 wait complete. 2023-10-06T02:45:23.794 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T02:45:23.808 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-06T02:45:23.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-06T02:45:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 Processor -- start 2023-10-06T02:45:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 -- start start 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdc6c0680c0 0x7fdc6c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c069140 con 0x7fdc6c0acd60 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0b3980 con 0x7fdc6c0680c0 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc71e8d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0b0270 con 0x7fdc6c159600 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdc6c0680c0 0x7fdc6c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdc6c0680c0 0x7fdc6c1590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36372/0 (socket says 172.21.15.33:36372) 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 -- 172.21.15.33:0/3235418977 learned_addr learned my addr 172.21.15.33:0/3235418977 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc7168c700 1 --2- 172.21.15.33:0/3235418977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 -- 172.21.15.33:0/3235418977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 msgr2=0x7fdc6c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 --2- 172.21.15.33:0/3235418977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 -- 172.21.15.33:0/3235418977 >> v1:172.21.15.167:6789/0 conn(0x7fdc6c0acd60 legacy=0x7fdc6c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 -- 172.21.15.33:0/3235418977 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c068530 con 0x7fdc6c0680c0 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.917+0000 7fdc6bfff700 1 --2- 172.21.15.33:0/3235418977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c1590c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc540099b0 tx=0x7fdc54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36547c6870162abc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6b7fe700 1 -- 172.21.15.33:0/3235418977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400b860 con 0x7fdc6c0680c0 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6b7fe700 1 -- 172.21.15.33:0/3235418977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5400b9c0 con 0x7fdc6c0680c0 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6b7fe700 1 -- 172.21.15.33:0/3235418977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400bd60 con 0x7fdc6c0680c0 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- 172.21.15.33:0/3235418977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0680c0 msgr2=0x7fdc6c1590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/3235418977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c1590c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc540099b0 tx=0x7fdc54009250 comp rx=0 tx=0).stop 2023-10-06T02:45:23.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- 172.21.15.33:0/3235418977 shutdown_connections 2023-10-06T02:45:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/3235418977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/3235418977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c1590c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- 172.21.15.33:0/3235418977 >> 172.21.15.33:0/3235418977 conn(0x7fdc6c06c3a0 msgr2=0x7fdc6c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- 172.21.15.33:0/3235418977 shutdown_connections 2023-10-06T02:45:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- 172.21.15.33:0/3235418977 wait complete. 2023-10-06T02:45:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 Processor -- start 2023-10-06T02:45:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- start start 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 0x7fdc6c149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1499d0 con 0x7fdc6c0acd60 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cbd80 con 0x7fdc6c0680c0 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc71e8d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cc060 con 0x7fdc6c159600 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 0x7fdc6c149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 0x7fdc6c149490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36382/0 (socket says 172.21.15.33:36382) 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc70e8b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 -- 172.21.15.33:0/612449100 learned_addr learned my addr 172.21.15.33:0/612449100 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc7168c700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 -- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 msgr2=0x7fdc6c0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 -- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6c0680c0 msgr2=0x7fdc6c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c0cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.921+0000 7fdc6bfff700 1 -- 172.21.15.33:0/612449100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc540094f0 con 0x7fdc6c0acd60 2023-10-06T02:45:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc6bfff700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 0x7fdc6c149490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009fd0 tx=0x7fdc54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54011e40 con 0x7fdc6c0acd60 2023-10-06T02:45:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc54024440 con 0x7fdc6c0acd60 2023-10-06T02:45:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c0cc340 con 0x7fdc6c0acd60 2023-10-06T02:45:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc540247e0 con 0x7fdc6c0acd60 2023-10-06T02:45:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c0cc900 con 0x7fdc6c0acd60 2023-10-06T02:45:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c071040 con 0x7fdc6c0acd60 2023-10-06T02:45:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdc5401a420 con 0x7fdc6c0acd60 2023-10-06T02:45:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc68ff9700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc4c060580 0x7fdc4c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.925+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdc5407edc0 con 0x7fdc6c0acd60 2023-10-06T02:45:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.929+0000 7fdc70e8b700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc4c060580 0x7fdc4c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.929+0000 7fdc70e8b700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc4c060580 0x7fdc4c062a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc64007fd0 tx=0x7fdc64000f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:23.933+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdc5404f230 con 0x7fdc6c0acd60 2023-10-06T02:45:24.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.105+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fdc6c068810 con 0x7fdc6c0acd60 2023-10-06T02:45:24.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.293+0000 7fdc68ff9700 1 -- 172.21.15.33:0/612449100 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v25) v1 ==== 157+0+0 (secure 0 0 0) 0x7fdc54052850 con 0x7fdc6c0acd60 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc4c060580 msgr2=0x7fdc4c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc4c060580 0x7fdc4c062a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc64007fd0 tx=0x7fdc64000f40 comp rx=0 tx=0).stop 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 msgr2=0x7fdc6c149490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 0x7fdc6c149490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009fd0 tx=0x7fdc54017000 comp rx=0 tx=0).stop 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 shutdown_connections 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc4c060580 0x7fdc4c062a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc6c159600 0x7fdc6c0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc6c0acd60 0x7fdc6c149490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 --2- 172.21.15.33:0/612449100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6c0680c0 0x7fdc6c0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 >> 172.21.15.33:0/612449100 conn(0x7fdc6c06c3a0 msgr2=0x7fdc6c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 shutdown_connections 2023-10-06T02:45:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.297+0000 7fdc71e8d700 1 -- 172.21.15.33:0/612449100 wait complete. 2023-10-06T02:45:24.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:45:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.429+0000 7fd699a8e700 1 Processor -- start 2023-10-06T02:45:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.429+0000 7fd699a8e700 1 -- start start 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd694156a40 0x7fd6940ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6940c8cc0 0x7fd6940b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd6940b1700 0x7fd69406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940cb800 con 0x7fd694156a40 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940b1b20 con 0x7fd6940b1700 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c9120 con 0x7fd6940c8cc0 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd694156a40 0x7fd6940ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd694156a40 0x7fd6940ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36388/0 (socket says 172.21.15.33:36388) 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 -- 172.21.15.33:0/698045289 learned_addr learned my addr 172.21.15.33:0/698045289 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd69928d700 1 --2- 172.21.15.33:0/698045289 >> v2:172.21.15.167:3300/0 conn(0x7fd6940b1700 0x7fd69406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd693fff700 1 --2- 172.21.15.33:0/698045289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6940c8cc0 0x7fd6940b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 -- 172.21.15.33:0/698045289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6940c8cc0 msgr2=0x7fd6940b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 --2- 172.21.15.33:0/698045289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6940c8cc0 0x7fd6940b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 -- 172.21.15.33:0/698045289 >> v2:172.21.15.167:3300/0 conn(0x7fd6940b1700 msgr2=0x7fd69406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 --2- 172.21.15.33:0/698045289 >> v2:172.21.15.167:3300/0 conn(0x7fd6940b1700 0x7fd69406ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 -- 172.21.15.33:0/698045289 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940cabf0 con 0x7fd694156a40 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd698a8c700 1 --2- 172.21.15.33:0/698045289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694156a40 0x7fd6940ca5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd69406dd40 tx=0x7fd688009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5002be4a895ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd6937fe700 1 -- 172.21.15.33:0/698045289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800b860 con 0x7fd694156a40 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd6937fe700 1 -- 172.21.15.33:0/698045289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800b9c0 con 0x7fd694156a40 2023-10-06T02:45:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd6937fe700 1 -- 172.21.15.33:0/698045289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800bd60 con 0x7fd694156a40 2023-10-06T02:45:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 -- 172.21.15.33:0/698045289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694156a40 msgr2=0x7fd6940ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- 172.21.15.33:0/698045289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694156a40 0x7fd6940ca5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd69406dd40 tx=0x7fd688009250 comp rx=0 tx=0).stop 2023-10-06T02:45:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 -- 172.21.15.33:0/698045289 shutdown_connections 2023-10-06T02:45:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- 172.21.15.33:0/698045289 >> v2:172.21.15.167:3300/0 conn(0x7fd6940b1700 0x7fd69406ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- 172.21.15.33:0/698045289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6940c8cc0 0x7fd6940b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.433+0000 7fd699a8e700 1 --2- 172.21.15.33:0/698045289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694156a40 0x7fd6940ca5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.437+0000 7fd699a8e700 1 -- 172.21.15.33:0/698045289 >> 172.21.15.33:0/698045289 conn(0x7fd69406a900 msgr2=0x7fd6940adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.437+0000 7fd699a8e700 1 -- 172.21.15.33:0/698045289 shutdown_connections 2023-10-06T02:45:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.437+0000 7fd699a8e700 1 -- 172.21.15.33:0/698045289 wait complete. 2023-10-06T02:45:24.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.437+0000 7fd699a8e700 1 Processor -- start 2023-10-06T02:45:24.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.437+0000 7fd699a8e700 1 -- start start 2023-10-06T02:45:24.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 0x7fd6940a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6940c8cc0 0x7fd694167c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd694156a40 0x7fd694168150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940a4a00 con 0x7fd6940b1700 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940a4c10 con 0x7fd6940c8cc0 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd694159250 con 0x7fd694156a40 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 0x7fd6940a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 0x7fd6940a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36392/0 (socket says 172.21.15.33:36392) 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 -- 172.21.15.33:0/4129816298 learned_addr learned my addr 172.21.15.33:0/4129816298 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd69928d700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd694156a40 0x7fd694168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd693fff700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6940c8cc0 0x7fd694167c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 -- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd694156a40 msgr2=0x7fd694168150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:24.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd694156a40 0x7fd694168150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 -- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6940c8cc0 msgr2=0x7fd694167c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:24.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6940c8cc0 0x7fd694167c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:24.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 -- 172.21.15.33:0/4129816298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6880094f0 con 0x7fd6940b1700 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd698a8c700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 0x7fd6940a44c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd69406aea0 tx=0x7fd688017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688024440 con 0x7fd6940b1700 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6880245a0 con 0x7fd6940b1700 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6941596e0 con 0x7fd6940b1700 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688024960 con 0x7fd6940b1700 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.441+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd694158c00 con 0x7fd6940b1700 2023-10-06T02:45:24.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.445+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69404be40 con 0x7fd6940b1700 2023-10-06T02:45:24.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.445+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd688024d50 con 0x7fd6940b1700 2023-10-06T02:45:24.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.445+0000 7fd690ff9700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd678060580 0x7fd678062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:24.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.445+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fd68807e630 con 0x7fd6940b1700 2023-10-06T02:45:24.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.445+0000 7fd693fff700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd678060580 0x7fd678062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:24.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.445+0000 7fd693fff700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd678060580 0x7fd678062a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd694151530 tx=0x7fd680008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:24.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.457+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd68804e9e0 con 0x7fd6940b1700 2023-10-06T02:45:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:24.617+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd6940caed0 con 0x7fd6940b1700 2023-10-06T02:45:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.305+0000 7fd690ff9700 1 -- 172.21.15.33:0/4129816298 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v27) v1 ==== 228+0+0 (secure 0 0 0) 0x7fd688052000 con 0x7fd6940b1700 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd678060580 msgr2=0x7fd678062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd678060580 0x7fd678062a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd694151530 tx=0x7fd680008040 comp rx=0 tx=0).stop 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 msgr2=0x7fd6940a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 0x7fd6940a44c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd69406aea0 tx=0x7fd688017000 comp rx=0 tx=0).stop 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 shutdown_connections 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd678060580 0x7fd678062a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd694156a40 0x7fd694168150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6940c8cc0 0x7fd694167c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 --2- 172.21.15.33:0/4129816298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6940b1700 0x7fd6940a44c0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd69406aea0 tx=0x7fd688017000 comp rx=0 tx=0).stop 2023-10-06T02:45:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 >> 172.21.15.33:0/4129816298 conn(0x7fd69406a900 msgr2=0x7fd694154140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 shutdown_connections 2023-10-06T02:45:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.321+0000 7fd699a8e700 1 -- 172.21.15.33:0/4129816298 wait complete. 2023-10-06T02:45:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' created 2023-10-06T02:45:26.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-06T02:45:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 Processor -- start 2023-10-06T02:45:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 -- start start 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bc40671e0 0x7f0bc40675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc40c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc406dd20 con 0x7f0bc406edc0 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc406e000 con 0x7f0bc40671e0 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc89a8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc406e2e0 con 0x7f0bc40c91e0 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bc40671e0 0x7f0bc40675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc359e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0bc406edc0 0x7f0bc415bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60298/0 (socket says 172.21.15.33:60298) 2023-10-06T02:45:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0bc40671e0 0x7f0bc40675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36394/0 (socket says 172.21.15.33:36394) 2023-10-06T02:45:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc359e700 1 -- 172.21.15.33:0/1017199659 learned_addr learned my addr 172.21.15.33:0/1017199659 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.441+0000 7f0bc259c700 1 --2- 172.21.15.33:0/1017199659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc40c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc2d9d700 1 -- 172.21.15.33:0/1017199659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 msgr2=0x7f0bc40c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc2d9d700 1 --2- 172.21.15.33:0/1017199659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc40c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc2d9d700 1 -- 172.21.15.33:0/1017199659 >> v1:172.21.15.33:6789/0 conn(0x7f0bc406edc0 legacy=0x7f0bc415bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc1d9b700 1 -- 172.21.15.33:0/1017199659 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342161094 0 0) 0x7f0bc406dd20 con 0x7f0bc406edc0 2023-10-06T02:45:26.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc2d9d700 1 -- 172.21.15.33:0/1017199659 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc406e5c0 con 0x7f0bc40671e0 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc2d9d700 1 --2- 172.21.15.33:0/1017199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc40671e0 0x7f0bc40675b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009e60 tx=0x7f0bb4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f44c69cbb84221c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc1d9b700 1 -- 172.21.15.33:0/1017199659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400e070 con 0x7f0bc40671e0 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc1d9b700 1 -- 172.21.15.33:0/1017199659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb400bbf0 con 0x7f0bc40671e0 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc1d9b700 1 -- 172.21.15.33:0/1017199659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4012430 con 0x7f0bc40671e0 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1017199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc40671e0 msgr2=0x7f0bc40675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1017199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc40671e0 0x7f0bc40675b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009e60 tx=0x7f0bb4009780 comp rx=0 tx=0).stop 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1017199659 shutdown_connections 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1017199659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc40c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1017199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc40671e0 0x7f0bc40675b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1017199659 >> 172.21.15.33:0/1017199659 conn(0x7f0bc40ccb30 msgr2=0x7f0bc40c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1017199659 shutdown_connections 2023-10-06T02:45:26.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1017199659 wait complete. 2023-10-06T02:45:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 Processor -- start 2023-10-06T02:45:26.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.445+0000 7f0bc89a8700 1 -- start start 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc40671e0 0x7f0bc4169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 0x7f0bc416c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc416a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40a98f0 con 0x7f0bc406edc0 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40af190 con 0x7f0bc40671e0 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40a9050 con 0x7f0bc40c91e0 2023-10-06T02:45:26.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 0x7f0bc416c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 0x7f0bc416c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36404/0 (socket says 172.21.15.33:36404) 2023-10-06T02:45:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 -- 172.21.15.33:0/1591914814 learned_addr learned my addr 172.21.15.33:0/1591914814 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc2d9d700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc40671e0 0x7f0bc4169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:26.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc359e700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc416a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:26.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 -- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 msgr2=0x7f0bc416a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:26.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc416a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 -- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc40671e0 msgr2=0x7f0bc4169c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:26.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc40671e0 0x7f0bc4169c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 -- 172.21.15.33:0/1591914814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb4009a20 con 0x7f0bc406edc0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc259c700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 0x7f0bc416c0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e980 tx=0x7f0bb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8013680 con 0x7f0bc406edc0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb80137e0 con 0x7f0bc406edc0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8013ba0 con 0x7f0bc406edc0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc40a9330 con 0x7f0bc406edc0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc40a88e0 con 0x7f0bc406edc0 2023-10-06T02:45:26.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.449+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc4071040 con 0x7f0bc406edc0 2023-10-06T02:45:26.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.453+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0bb8026430 con 0x7f0bc406edc0 2023-10-06T02:45:26.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.453+0000 7f0baaffd700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0bb0060550 0x7f0bb0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:26.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.453+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f0bb807d510 con 0x7f0bc406edc0 2023-10-06T02:45:26.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.453+0000 7f0bc2d9d700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0bb0060550 0x7f0bb0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:26.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.453+0000 7f0bc2d9d700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0bb0060550 0x7f0bb0062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40097c0 tx=0x7f0bb4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.461+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0bb804d6b0 con 0x7f0bc406edc0 2023-10-06T02:45:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:26.621+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f0bc40af2d0 con 0x7f0bc406edc0 2023-10-06T02:45:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.317+0000 7f0baaffd700 1 -- 172.21.15.33:0/1591914814 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0bb8050cd0 con 0x7f0bc406edc0 2023-10-06T02:45:27.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0bb0060550 msgr2=0x7f0bb0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:27.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0bb0060550 0x7f0bb0062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40097c0 tx=0x7f0bb4019000 comp rx=0 tx=0).stop 2023-10-06T02:45:27.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 msgr2=0x7f0bc416c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:27.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 0x7f0bc416c0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e980 tx=0x7f0bb80091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 shutdown_connections 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0bb0060550 0x7f0bb0062a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bc40c91e0 0x7f0bc416a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc406edc0 0x7f0bc416c0a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.333+0000 7f0bc89a8700 1 --2- 172.21.15.33:0/1591914814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bc40671e0 0x7f0bc4169c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.337+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 >> 172.21.15.33:0/1591914814 conn(0x7f0bc40ccb30 msgr2=0x7f0bc40ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.337+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 shutdown_connections 2023-10-06T02:45:27.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.337+0000 7f0bc89a8700 1 -- 172.21.15.33:0/1591914814 wait complete. 2023-10-06T02:45:27.342 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 5 allow_ec_overwrites to true 2023-10-06T02:45:27.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-06T02:45:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.453+0000 7fa24359e700 1 Processor -- start 2023-10-06T02:45:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.453+0000 7fa24359e700 1 -- start start 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa244068aa0 0x7fa2440b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa2440b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2440ce580 con 0x7fa244158c60 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2440cf620 con 0x7fa244068aa0 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440ce860 con 0x7fa2440cf7c0 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa244068aa0 0x7fa2440b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa244068aa0 0x7fa2440b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36412/0 (socket says 172.21.15.33:36412) 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 -- 172.21.15.33:0/3003305698 learned_addr learned my addr 172.21.15.33:0/3003305698 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa242d9d700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa2440b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 -- 172.21.15.33:0/3003305698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 msgr2=0x7fa2440b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa2440b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 -- 172.21.15.33:0/3003305698 >> v1:172.21.15.167:6789/0 conn(0x7fa244158c60 legacy=0x7fa244068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 -- 172.21.15.33:0/3003305698 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2440ceb40 con 0x7fa244068aa0 2023-10-06T02:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa242d9d700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa2440b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa239d9b700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244068aa0 0x7fa2440b29e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0099b0 tx=0x7fa22c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736f956677b661c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa241d9b700 1 -- 172.21.15.33:0/3003305698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00b860 con 0x7fa244068aa0 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa241d9b700 1 -- 172.21.15.33:0/3003305698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c00b9c0 con 0x7fa244068aa0 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa241d9b700 1 -- 172.21.15.33:0/3003305698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00bd60 con 0x7fa244068aa0 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- 172.21.15.33:0/3003305698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244068aa0 msgr2=0x7fa2440b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244068aa0 0x7fa2440b29e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0099b0 tx=0x7fa22c009250 comp rx=0 tx=0).stop 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- 172.21.15.33:0/3003305698 shutdown_connections 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa2440b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 --2- 172.21.15.33:0/3003305698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244068aa0 0x7fa2440b29e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- 172.21.15.33:0/3003305698 >> 172.21.15.33:0/3003305698 conn(0x7fa244067a40 msgr2=0x7fa2440b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- 172.21.15.33:0/3003305698 shutdown_connections 2023-10-06T02:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.457+0000 7fa24359e700 1 -- 172.21.15.33:0/3003305698 wait complete. 2023-10-06T02:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 Processor -- start 2023-10-06T02:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 -- start start 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa24414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 0x7fa24414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2440b50f0 0x7fa2440b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24414df50 con 0x7fa244158c60 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440b4c60 con 0x7fa2440b50f0 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440b4050 con 0x7fa2440cf7c0 2023-10-06T02:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 0x7fa24414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa24414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa24414f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:45794/0 (socket says 172.21.15.33:45794) 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa24259c700 1 -- 172.21.15.33:0/2166799513 learned_addr learned my addr 172.21.15.33:0/2166799513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa242d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2440b50f0 0x7fa2440b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 0x7fa24414da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36422/0 (socket says 172.21.15.33:36422) 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 -- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 msgr2=0x7fa24414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa24414f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 -- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2440b50f0 msgr2=0x7fa2440b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2440b50f0 0x7fa2440b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 -- 172.21.15.33:0/2166799513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c0094f0 con 0x7fa244158c60 2023-10-06T02:45:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa242d9d700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2440b50f0 0x7fa2440b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.461+0000 7fa239d9b700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 0x7fa24414da10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa24406ff30 tx=0x7fa22c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00b860 con 0x7fa244158c60 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c01f070 con 0x7fa244158c60 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2440b4330 con 0x7fa244158c60 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c026480 con 0x7fa244158c60 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2440b48f0 con 0x7fa244158c60 2023-10-06T02:45:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa22c01a430 con 0x7fa244158c60 2023-10-06T02:45:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2440b4dd0 con 0x7fa244158c60 2023-10-06T02:45:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.465+0000 7fa23affd700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa224060550 0x7fa224062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.469+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fa22c021070 con 0x7fa244158c60 2023-10-06T02:45:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.469+0000 7fa24259c700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa224060550 0x7fa224062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.469+0000 7fa24259c700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa224060550 0x7fa224062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa234003ab0 tx=0x7fa234001090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.473+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa22c04e4d0 con 0x7fa244158c60 2023-10-06T02:45:27.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:27.673+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fa244071040 con 0x7fa244158c60 2023-10-06T02:45:28.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.321+0000 7fa23affd700 1 -- 172.21.15.33:0/2166799513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fa22c051af0 con 0x7fa244158c60 2023-10-06T02:45:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa224060550 msgr2=0x7fa224062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa224060550 0x7fa224062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa234003ab0 tx=0x7fa234001090 comp rx=0 tx=0).stop 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 msgr2=0x7fa24414da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 0x7fa24414da10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa24406ff30 tx=0x7fa22c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 shutdown_connections 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2440b50f0 0x7fa2440b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa244158c60 0x7fa24414da10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa224060550 0x7fa224062a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 --2- 172.21.15.33:0/2166799513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2440cf7c0 0x7fa24414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 >> 172.21.15.33:0/2166799513 conn(0x7fa244067a40 msgr2=0x7fa244153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 shutdown_connections 2023-10-06T02:45:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.325+0000 7fa24359e700 1 -- 172.21.15.33:0/2166799513 wait complete. 2023-10-06T02:45:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:added data pool 5 to fsmap 2023-10-06T02:45:28.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 Processor -- start 2023-10-06T02:45:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 -- start start 2023-10-06T02:45:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e806b050 0x7f35e8066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f35e80c93a0 0x7f35e80c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e806b420 con 0x7f35e80c6610 2023-10-06T02:45:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e80674f0 con 0x7f35e80c93a0 2023-10-06T02:45:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35ec8f2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e806a4e0 con 0x7f35e806b050 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e806b050 0x7f35e8066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f35e80c93a0 0x7f35e80c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f35e80c93a0 0x7f35e80c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44080/0 (socket says 172.21.15.33:44080) 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e806b050 0x7f35e8066fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:45802/0 (socket says 172.21.15.33:45802) 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 -- 172.21.15.33:0/2089051 learned_addr learned my addr 172.21.15.33:0/2089051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 -- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e806b050 msgr2=0x7f35e8066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 --2- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e806b050 0x7f35e8066fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 -- 172.21.15.33:0/2089051 >> v1:172.21.15.167:6789/0 conn(0x7f35e80c6610 legacy=0x7f35e80c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 -- 172.21.15.33:0/2089051 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e8067700 con 0x7f35e80c93a0 2023-10-06T02:45:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e67fc700 1 -- 172.21.15.33:0/2089051 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094201654 0 0) 0x7f35e806b420 con 0x7f35e80c6610 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.461+0000 7f35e6ffd700 1 --2- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e80c60d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35d00099b0 tx=0x7f35d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cbebd2469ecb930 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e67fc700 1 -- 172.21.15.33:0/2089051 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0009ea0 con 0x7f35e80c93a0 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e67fc700 1 -- 172.21.15.33:0/2089051 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000b860 con 0x7f35e80c93a0 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e67fc700 1 -- 172.21.15.33:0/2089051 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000bc00 con 0x7f35e80c93a0 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 msgr2=0x7f35e80c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e80c60d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35d00099b0 tx=0x7f35d0009250 comp rx=0 tx=0).stop 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- 172.21.15.33:0/2089051 shutdown_connections 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e80c60d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/2089051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e806b050 0x7f35e8066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- 172.21.15.33:0/2089051 >> 172.21.15.33:0/2089051 conn(0x7f35e80c8240 msgr2=0x7f35e80c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- 172.21.15.33:0/2089051 shutdown_connections 2023-10-06T02:45:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- 172.21.15.33:0/2089051 wait complete. 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 Processor -- start 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- start start 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 0x7f35e815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e80c6610 0x7f35e815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80af5a0 con 0x7f35e806b050 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80af880 con 0x7f35e80c93a0 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35ec8f2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e81620d0 con 0x7f35e80c6610 2023-10-06T02:45:28.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 0x7f35e815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 0x7f35e815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36432/0 (socket says 172.21.15.33:36432) 2023-10-06T02:45:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e77fe700 1 -- 172.21.15.33:0/496214051 learned_addr learned my addr 172.21.15.33:0/496214051 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e6ffd700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e80c6610 0x7f35e815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.465+0000 7f35e7fff700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35e77fe700 1 -- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e80c6610 msgr2=0x7f35e815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35e77fe700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e80c6610 0x7f35e815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35e77fe700 1 -- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 msgr2=0x7f35e815fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35e77fe700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e815fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35e77fe700 1 -- 172.21.15.33:0/496214051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00094f0 con 0x7f35e806b050 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35e77fe700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 0x7f35e815aa30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f35d800e960 tx=0x7f35d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8013680 con 0x7f35e806b050 2023-10-06T02:45:28.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80137e0 con 0x7f35e806b050 2023-10-06T02:45:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e81623b0 con 0x7f35e806b050 2023-10-06T02:45:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8013b80 con 0x7f35e806b050 2023-10-06T02:45:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e8161c10 con 0x7f35e806b050 2023-10-06T02:45:28.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e8161e60 con 0x7f35e806b050 2023-10-06T02:45:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.469+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f35d80123d0 con 0x7f35e806b050 2023-10-06T02:45:28.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.473+0000 7f35df7fe700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f35c8060550 0x7f35c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.473+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f35d807cc90 con 0x7f35e806b050 2023-10-06T02:45:28.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.473+0000 7f35e6ffd700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f35c8060550 0x7f35c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.473+0000 7f35e6ffd700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f35c8060550 0x7f35c8062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35d00099b0 tx=0x7f35d0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.481+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f35d804ce30 con 0x7f35e806b050 2023-10-06T02:45:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.685+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35e8071040 con 0x7f35e806b050 2023-10-06T02:45:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.685+0000 7f35df7fe700 1 -- 172.21.15.33:0/496214051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f35d8017090 con 0x7f35e806b050 2023-10-06T02:45:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:28.691 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:28.315508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T02:45:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f35c8060550 msgr2=0x7f35c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f35c8060550 0x7f35c8062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35d00099b0 tx=0x7f35d0017000 comp rx=0 tx=0).stop 2023-10-06T02:45:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 msgr2=0x7f35e815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 0x7f35e815aa30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f35d800e960 tx=0x7f35d80091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 shutdown_connections 2023-10-06T02:45:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f35c8060550 0x7f35c8062a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35e80c6610 0x7f35e815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35e80c93a0 0x7f35e815fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 --2- 172.21.15.33:0/496214051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e806b050 0x7f35e815aa30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 >> 172.21.15.33:0/496214051 conn(0x7f35e80c8240 msgr2=0x7f35e815c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 shutdown_connections 2023-10-06T02:45:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.689+0000 7f35ec8f2700 1 -- 172.21.15.33:0/496214051 wait complete. 2023-10-06T02:45:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-10-06T02:45:28.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:28.766 INFO:tasks.ceph.mds.c.smithi033.stderr:2023-10-06T02:45:28.761+0000 7f3acdbea700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:45:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 Processor -- start 2023-10-06T02:45:28.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- start start 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcd40069480 0x7fcd4015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd400cc5b0 0x7fcd40159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4006bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4015a330 con 0x7fcd400cc5b0 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd40159d70 con 0x7fcd40069480 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4015a050 con 0x7fcd40159850 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd400cc5b0 0x7fcd40159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd400cc5b0 0x7fcd40159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36444/0 (socket says 172.21.15.33:36444) 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 -- 172.21.15.33:0/1047798891 learned_addr learned my addr 172.21.15.33:0/1047798891 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd44e01700 1 --2- 172.21.15.33:0/1047798891 >> v2:172.21.15.167:3300/0 conn(0x7fcd40069480 0x7fcd4015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 -- 172.21.15.33:0/1047798891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 msgr2=0x7fcd4006bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd45602700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4006bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4006bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 -- 172.21.15.33:0/1047798891 >> v2:172.21.15.167:3300/0 conn(0x7fcd40069480 msgr2=0x7fcd4015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 --2- 172.21.15.33:0/1047798891 >> v2:172.21.15.167:3300/0 conn(0x7fcd40069480 0x7fcd4015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 -- 172.21.15.33:0/1047798891 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4015cac0 con 0x7fcd400cc5b0 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd44e01700 1 --2- 172.21.15.33:0/1047798891 >> v2:172.21.15.167:3300/0 conn(0x7fcd40069480 0x7fcd4015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd45602700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4006bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3ffff700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400cc5b0 0x7fcd40159310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd4006bb10 tx=0x7fcd28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db684027c51dd8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:28.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3f7fe700 1 -- 172.21.15.33:0/1047798891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800e070 con 0x7fcd400cc5b0 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3f7fe700 1 -- 172.21.15.33:0/1047798891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2800ba50 con 0x7fcd400cc5b0 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd3f7fe700 1 -- 172.21.15.33:0/1047798891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280092e0 con 0x7fcd400cc5b0 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- 172.21.15.33:0/1047798891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400cc5b0 msgr2=0x7fcd40159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400cc5b0 0x7fcd40159310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd4006bb10 tx=0x7fcd28009840 comp rx=0 tx=0).stop 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- 172.21.15.33:0/1047798891 shutdown_connections 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4006bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- 172.21.15.33:0/1047798891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd400cc5b0 0x7fcd40159310 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 --2- 172.21.15.33:0/1047798891 >> v2:172.21.15.167:3300/0 conn(0x7fcd40069480 0x7fcd4015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- 172.21.15.33:0/1047798891 >> 172.21.15.33:0/1047798891 conn(0x7fcd4006e7c0 msgr2=0x7fcd4006ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- 172.21.15.33:0/1047798891 shutdown_connections 2023-10-06T02:45:28.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.809+0000 7fcd47065700 1 -- 172.21.15.33:0/1047798891 wait complete. 2023-10-06T02:45:28.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.813+0000 7fcd47065700 1 Processor -- start 2023-10-06T02:45:28.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.813+0000 7fcd47065700 1 -- start start 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.813+0000 7fcd47065700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 0x7fcd40155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd400cc5b0 0x7fcd4015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 0x7fcd40155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40155b60 con 0x7fcd40069480 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 0x7fcd40155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36448/0 (socket says 172.21.15.33:36448) 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400a9d20 con 0x7fcd400cc5b0 2023-10-06T02:45:28.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 -- 172.21.15.33:0/206204752 learned_addr learned my addr 172.21.15.33:0/206204752 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400b0480 con 0x7fcd40159850 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd45602700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd3ffff700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd400cc5b0 0x7fcd4015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 -- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 msgr2=0x7fcd4015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 -- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd400cc5b0 msgr2=0x7fcd4015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd400cc5b0 0x7fcd4015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 -- 172.21.15.33:0/206204752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd28009ae0 con 0x7fcd40069480 2023-10-06T02:45:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd44e01700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 0x7fcd40155620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800c880 tx=0x7fcd38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011600 con 0x7fcd40069480 2023-10-06T02:45:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38011760 con 0x7fcd40069480 2023-10-06T02:45:28.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011b20 con 0x7fcd40069480 2023-10-06T02:45:28.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd400b0760 con 0x7fcd40069480 2023-10-06T02:45:28.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd400b0d20 con 0x7fcd40069480 2023-10-06T02:45:28.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd400a9b80 con 0x7fcd40069480 2023-10-06T02:45:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.817+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcd380193b0 con 0x7fcd40069480 2023-10-06T02:45:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.821+0000 7fcd3cff9700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcd20060550 0x7fcd20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.821+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcd3807cb70 con 0x7fcd40069480 2023-10-06T02:45:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.821+0000 7fcd3ffff700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcd20060550 0x7fcd20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:28.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.821+0000 7fcd3ffff700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcd20060550 0x7fcd20062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd4006bb10 tx=0x7fcd28018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:28.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.825+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcd400a9b80 con 0x7fcd40069480 2023-10-06T02:45:28.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.989+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd400a9b80 con 0x7fcd40069480 2023-10-06T02:45:28.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.989+0000 7fcd3cff9700 1 -- 172.21.15.33:0/206204752 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fcd400a9b80 con 0x7fcd40069480 2023-10-06T02:45:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:28.996 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcd20060550 msgr2=0x7fcd20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcd20060550 0x7fcd20062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd4006bb10 tx=0x7fcd28018000 comp rx=0 tx=0).stop 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 msgr2=0x7fcd40155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 0x7fcd40155620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800c880 tx=0x7fcd38008040 comp rx=0 tx=0).stop 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 shutdown_connections 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcd20060550 0x7fcd20062a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd40159850 0x7fcd4015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd400cc5b0 0x7fcd4015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 --2- 172.21.15.33:0/206204752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd40069480 0x7fcd40155620 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 >> 172.21.15.33:0/206204752 conn(0x7fcd4006e7c0 msgr2=0x7fcd40156380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 shutdown_connections 2023-10-06T02:45:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:28.993+0000 7fcd47065700 1 -- 172.21.15.33:0/206204752 wait complete. 2023-10-06T02:45:29.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 Processor -- start 2023-10-06T02:45:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 -- start start 2023-10-06T02:45:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaac0ade00 0x7faaac0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faaac066fb0 0x7faaac0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac06add0 con 0x7faaac0c5940 2023-10-06T02:45:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaac0c80c0 con 0x7faaac0ade00 2023-10-06T02:45:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faab0c25700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaac0c7c40 con 0x7faaac066fb0 2023-10-06T02:45:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaac0ade00 0x7faaac0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaac0ade00 0x7faaac0c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36458/0 (socket says 172.21.15.33:36458) 2023-10-06T02:45:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 -- 172.21.15.33:0/3011157295 learned_addr learned my addr 172.21.15.33:0/3011157295 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaabfff700 1 --2- 172.21.15.33:0/3011157295 >> v2:172.21.15.167:3300/0 conn(0x7faaac066fb0 0x7faaac0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 -- 172.21.15.33:0/3011157295 >> v2:172.21.15.167:3300/0 conn(0x7faaac066fb0 msgr2=0x7faaac0c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 --2- 172.21.15.33:0/3011157295 >> v2:172.21.15.167:3300/0 conn(0x7faaac066fb0 0x7faaac0c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 -- 172.21.15.33:0/3011157295 >> v1:172.21.15.167:6789/0 conn(0x7faaac0c5940 legacy=0x7faaac0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 -- 172.21.15.33:0/3011157295 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac0ae1d0 con 0x7faaac0ade00 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaab7fe700 1 --2- 172.21.15.33:0/3011157295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0ade00 0x7faaac0c71c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faaac1593f0 tx=0x7faa9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d4f9f1d77882f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.109+0000 7faaaa7fc700 1 -- 172.21.15.33:0/3011157295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c009e70 con 0x7faaac0ade00 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faaaa7fc700 1 -- 172.21.15.33:0/3011157295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c00b860 con 0x7faaac0ade00 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faaaa7fc700 1 -- 172.21.15.33:0/3011157295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00bc00 con 0x7faaac0ade00 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 -- 172.21.15.33:0/3011157295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0ade00 msgr2=0x7faaac0c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 --2- 172.21.15.33:0/3011157295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0ade00 0x7faaac0c71c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faaac1593f0 tx=0x7faa9c009250 comp rx=0 tx=0).stop 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 -- 172.21.15.33:0/3011157295 shutdown_connections 2023-10-06T02:45:29.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 --2- 172.21.15.33:0/3011157295 >> v2:172.21.15.167:3300/0 conn(0x7faaac066fb0 0x7faaac0c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 --2- 172.21.15.33:0/3011157295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0ade00 0x7faaac0c71c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 -- 172.21.15.33:0/3011157295 >> 172.21.15.33:0/3011157295 conn(0x7faaac0c8240 msgr2=0x7faaac0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:29.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 -- 172.21.15.33:0/3011157295 shutdown_connections 2023-10-06T02:45:29.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 -- 172.21.15.33:0/3011157295 wait complete. 2023-10-06T02:45:29.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 Processor -- start 2023-10-06T02:45:29.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.113+0000 7faab0c25700 1 -- start start 2023-10-06T02:45:29.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaac066fb0 0x7faaac0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaac0ade00 0x7faaac0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 0x7faaac149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0c5780 con 0x7faaac0c5940 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac14cb50 con 0x7faaac066fb0 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0c4f70 con 0x7faaac0ade00 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 0x7faaac149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 0x7faaac149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36464/0 (socket says 172.21.15.33:36464) 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 -- 172.21.15.33:0/3868871650 learned_addr learned my addr 172.21.15.33:0/3868871650 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaab7fe700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaac066fb0 0x7faaac0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaaaffd700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaac0ade00 0x7faaac0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 -- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaac0ade00 msgr2=0x7faaac0c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaac0ade00 0x7faaac0c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 -- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaac066fb0 msgr2=0x7faaac0c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaac066fb0 0x7faaac0c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 -- 172.21.15.33:0/3868871650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c0094f0 con 0x7faaac0c5940 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaabfff700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 0x7faaac149000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009e20 tx=0x7faaa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400caa0 con 0x7faaac0c5940 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa400f040 con 0x7faaac0c5940 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac0c3470 con 0x7faaac0c5940 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400ce70 con 0x7faaac0c5940 2023-10-06T02:45:29.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.117+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac0c54a0 con 0x7faaac0c5940 2023-10-06T02:45:29.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.121+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac0c7aa0 con 0x7faaac0c5940 2023-10-06T02:45:29.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.121+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faaa4014690 con 0x7faaac0c5940 2023-10-06T02:45:29.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.121+0000 7faaa37fe700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faa8c060580 0x7faa8c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.121+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7faaa407d650 con 0x7faaac0c5940 2023-10-06T02:45:29.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.121+0000 7faaab7fe700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faa8c060580 0x7faa8c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.121+0000 7faaab7fe700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faa8c060580 0x7faa8c062a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faaac1593f0 tx=0x7faa9c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.129+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faaac0c7aa0 con 0x7faaac0c5940 2023-10-06T02:45:29.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.289+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaac14cd00 con 0x7faaac0c5940 2023-10-06T02:45:29.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.289+0000 7faaa37fe700 1 -- 172.21.15.33:0/3868871650 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7faaac14cd00 con 0x7faaac0c5940 2023-10-06T02:45:29.296 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:29.296 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faa8c060580 msgr2=0x7faa8c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faa8c060580 0x7faa8c062a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faaac1593f0 tx=0x7faa9c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 msgr2=0x7faaac149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 0x7faaac149000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009e20 tx=0x7faaa400b040 comp rx=0 tx=0).stop 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 shutdown_connections 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faa8c060580 0x7faa8c062a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faaac0ade00 0x7faaac0c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaac0c5940 0x7faaac149000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 --2- 172.21.15.33:0/3868871650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faaac066fb0 0x7faaac0c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 >> 172.21.15.33:0/3868871650 conn(0x7faaac0c8240 msgr2=0x7faaac14e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 shutdown_connections 2023-10-06T02:45:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.293+0000 7faab0c25700 1 -- 172.21.15.33:0/3868871650 wait complete. 2023-10-06T02:45:29.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 Processor -- start 2023-10-06T02:45:29.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 -- start start 2023-10-06T02:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcc380ade00 0x7fcc380c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc380c5940 0x7fcc380c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc38066fb0 0x7fcc380c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc380c80c0 con 0x7fcc380c5940 2023-10-06T02:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3806add0 con 0x7fcc380ade00 2023-10-06T02:45:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc37fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380c7c40 con 0x7fcc38066fb0 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc367fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc380c5940 0x7fcc380c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc367fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc380c5940 0x7fcc380c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36466/0 (socket says 172.21.15.33:36466) 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc367fc700 1 -- 172.21.15.33:0/134570551 learned_addr learned my addr 172.21.15.33:0/134570551 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc377fe700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc38066fb0 0x7fcc380c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.429+0000 7fcc36ffd700 1 --2- 172.21.15.33:0/134570551 >> v2:172.21.15.167:3300/0 conn(0x7fcc380ade00 0x7fcc380c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc367fc700 1 -- 172.21.15.33:0/134570551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc38066fb0 msgr2=0x7fcc380c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc367fc700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc38066fb0 0x7fcc380c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc367fc700 1 -- 172.21.15.33:0/134570551 >> v2:172.21.15.167:3300/0 conn(0x7fcc380ade00 msgr2=0x7fcc380c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc367fc700 1 --2- 172.21.15.33:0/134570551 >> v2:172.21.15.167:3300/0 conn(0x7fcc380ade00 0x7fcc380c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc367fc700 1 -- 172.21.15.33:0/134570551 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc380ae1d0 con 0x7fcc380c5940 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc36ffd700 1 --2- 172.21.15.33:0/134570551 >> v2:172.21.15.167:3300/0 conn(0x7fcc380ade00 0x7fcc380c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc377fe700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc38066fb0 0x7fcc380c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:29.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc367fc700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc380c5940 0x7fcc380c7700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc380676c0 tx=0x7fcc20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bcd282d44e38c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc35ffb700 1 -- 172.21.15.33:0/134570551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000e070 con 0x7fcc380c5940 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc35ffb700 1 -- 172.21.15.33:0/134570551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000ba40 con 0x7fcc380c5940 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc35ffb700 1 -- 172.21.15.33:0/134570551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200092e0 con 0x7fcc380c5940 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 -- 172.21.15.33:0/134570551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc380c5940 msgr2=0x7fcc380c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc380c5940 0x7fcc380c7700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc380676c0 tx=0x7fcc20009840 comp rx=0 tx=0).stop 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 -- 172.21.15.33:0/134570551 shutdown_connections 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc38066fb0 0x7fcc380c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 --2- 172.21.15.33:0/134570551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc380c5940 0x7fcc380c7700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 --2- 172.21.15.33:0/134570551 >> v2:172.21.15.167:3300/0 conn(0x7fcc380ade00 0x7fcc380c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 -- 172.21.15.33:0/134570551 >> 172.21.15.33:0/134570551 conn(0x7fcc380c8240 msgr2=0x7fcc380c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 -- 172.21.15.33:0/134570551 shutdown_connections 2023-10-06T02:45:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.433+0000 7fcc37fff700 1 -- 172.21.15.33:0/134570551 wait complete. 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 Processor -- start 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 -- start start 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 0x7fcc380c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc380ade00 0x7fcc38146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc380c5940 0x7fcc3814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3816a4d0 con 0x7fcc38066fb0 2023-10-06T02:45:29.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3814c9a0 con 0x7fcc380ade00 2023-10-06T02:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3814cc80 con 0x7fcc380c5940 2023-10-06T02:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 0x7fcc380c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 0x7fcc380c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36474/0 (socket says 172.21.15.33:36474) 2023-10-06T02:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 -- 172.21.15.33:0/455778322 learned_addr learned my addr 172.21.15.33:0/455778322 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:29.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc367fc700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc380ade00 0x7fcc38146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc377fe700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc380c5940 0x7fcc3814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 -- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc380c5940 msgr2=0x7fcc3814af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc380c5940 0x7fcc3814af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 -- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc380ade00 msgr2=0x7fcc38146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc380ade00 0x7fcc38146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 -- 172.21.15.33:0/455778322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc20009ae0 con 0x7fcc38066fb0 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc36ffd700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 0x7fcc380c4ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800c830 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc28011600 con 0x7fcc38066fb0 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc28011760 con 0x7fcc38066fb0 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc28011b20 con 0x7fcc38066fb0 2023-10-06T02:45:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.437+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3814cf60 con 0x7fcc38066fb0 2023-10-06T02:45:29.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.441+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc380c5010 con 0x7fcc38066fb0 2023-10-06T02:45:29.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.441+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3814d1a0 con 0x7fcc38066fb0 2023-10-06T02:45:29.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.441+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcc28018360 con 0x7fcc38066fb0 2023-10-06T02:45:29.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.441+0000 7fcc2effd700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcc18060580 0x7fcc18062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.441+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcc2807cb80 con 0x7fcc38066fb0 2023-10-06T02:45:29.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.441+0000 7fcc367fc700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcc18060580 0x7fcc18062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.445+0000 7fcc367fc700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcc18060580 0x7fcc18062a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc380c5d80 tx=0x7fcc20018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.453+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcc3814d1a0 con 0x7fcc38066fb0 2023-10-06T02:45:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.645+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc3814d1a0 con 0x7fcc38066fb0 2023-10-06T02:45:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.645+0000 7fcc2effd700 1 -- 172.21.15.33:0/455778322 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fcc3814d1a0 con 0x7fcc38066fb0 2023-10-06T02:45:29.652 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:29.652 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:28.315508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-06T02:45:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcc18060580 msgr2=0x7fcc18062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcc18060580 0x7fcc18062a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc380c5d80 tx=0x7fcc20018000 comp rx=0 tx=0).stop 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 msgr2=0x7fcc380c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 0x7fcc380c4ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800c830 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 shutdown_connections 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcc18060580 0x7fcc18062a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc380c5940 0x7fcc3814af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc380ade00 0x7fcc38146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 --2- 172.21.15.33:0/455778322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc38066fb0 0x7fcc380c4ad0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 >> 172.21.15.33:0/455778322 conn(0x7fcc380c8240 msgr2=0x7fcc380c0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 shutdown_connections 2023-10-06T02:45:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.649+0000 7fcc37fff700 1 -- 172.21.15.33:0/455778322 wait complete. 2023-10-06T02:45:29.656 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2023-10-06T02:45:29.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.769+0000 7fbcce0cd700 1 Processor -- start 2023-10-06T02:45:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.769+0000 7fbcce0cd700 1 -- start start 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcc8155120 0x7fbcc8069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc80c7cc0 0x7fbcc806a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc80c7290 con 0x7fbcc81554f0 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc8156570 con 0x7fbcc8155120 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8155960 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcc8155120 0x7fbcc8069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbcc8155120 0x7fbcc8069dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36476/0 (socket says 172.21.15.33:36476) 2023-10-06T02:45:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1613330099 learned_addr learned my addr 172.21.15.33:0/1613330099 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccc8ca700 1 --1- 172.21.15.33:0/1613330099 >> v1:172.21.15.33:6789/0 conn(0x7fbcc81554f0 0x7fbcc80c75b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60308/0 (socket says 172.21.15.33:60308) 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd8cc700 1 --2- 172.21.15.33:0/1613330099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc80c7cc0 0x7fbcc806a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1613330099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc80c7cc0 msgr2=0x7fbcc806a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 --2- 172.21.15.33:0/1613330099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc80c7cc0 0x7fbcc806a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1613330099 >> v1:172.21.15.33:6789/0 conn(0x7fbcc81554f0 legacy=0x7fbcc80c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1613330099 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8155c40 con 0x7fbcc8155120 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbccd0cb700 1 --2- 172.21.15.33:0/1613330099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8155120 0x7fbcc8069dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40099b0 tx=0x7fbcc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec091e1fe510d980 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcbffff700 1 -- 172.21.15.33:0/1613330099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400b860 con 0x7fbcc8155120 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcbffff700 1 -- 172.21.15.33:0/1613330099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc400b9c0 con 0x7fbcc8155120 2023-10-06T02:45:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcbffff700 1 -- 172.21.15.33:0/1613330099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400bd60 con 0x7fbcc8155120 2023-10-06T02:45:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1613330099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8155120 msgr2=0x7fbcc8069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1613330099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8155120 0x7fbcc8069dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40099b0 tx=0x7fbcc4009250 comp rx=0 tx=0).stop 2023-10-06T02:45:29.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.773+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1613330099 shutdown_connections 2023-10-06T02:45:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1613330099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc80c7cc0 0x7fbcc806a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1613330099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc8155120 0x7fbcc8069dc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1613330099 >> 172.21.15.33:0/1613330099 conn(0x7fbcc80c8ae0 msgr2=0x7fbcc80c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1613330099 shutdown_connections 2023-10-06T02:45:29.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1613330099 wait complete. 2023-10-06T02:45:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 Processor -- start 2023-10-06T02:45:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- start start 2023-10-06T02:45:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 0x7fbcc8144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc8155120 0x7fbcc80c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc81554f0 0x7fbcc80ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81449b0 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80c0540 con 0x7fbcc81554f0 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbcce0cd700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80c0820 con 0x7fbcc8155120 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 0x7fbcc8144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 0x7fbcc8144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36482/0 (socket says 172.21.15.33:36482) 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1563814612 learned_addr learned my addr 172.21.15.33:0/1563814612 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd8cc700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc81554f0 0x7fbcc80ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccc8ca700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc8155120 0x7fbcc80c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc8155120 msgr2=0x7fbcc80c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc8155120 0x7fbcc80c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc81554f0 msgr2=0x7fbcc80ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc81554f0 0x7fbcc80ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccd0cb700 1 -- 172.21.15.33:0/1563814612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc40094f0 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.777+0000 7fbccc8ca700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc8155120 0x7fbcc80c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbccd0cb700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 0x7fbcc8144470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4006fd0 tx=0x7fbcc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc401a420 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc401a5a0 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc80c0b00 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc401a960 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc80c10c0 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc8071040 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbcc401ad10 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcbd7fa700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbcac060550 0x7fbcac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbcc407e720 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.781+0000 7fbccc8ca700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbcac060550 0x7fbcac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.785+0000 7fbccc8ca700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbcac060550 0x7fbcac062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbcc805e8a0 tx=0x7fbcb8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:29.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.789+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbcc404e8c0 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.949+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcc80c9900 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.953+0000 7fbcbd7fa700 1 -- 172.21.15.33:0/1563814612 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fbcc4051ee0 con 0x7fbcc80c7cc0 2023-10-06T02:45:29.957 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:29.957 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbcac060550 msgr2=0x7fbcac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbcac060550 0x7fbcac062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbcc805e8a0 tx=0x7fbcb8009040 comp rx=0 tx=0).stop 2023-10-06T02:45:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 msgr2=0x7fbcc8144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 0x7fbcc8144470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4006fd0 tx=0x7fbcc4017000 comp rx=0 tx=0).stop 2023-10-06T02:45:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 shutdown_connections 2023-10-06T02:45:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbcac060550 0x7fbcac062a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcc8155120 0x7fbcc80c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc81554f0 0x7fbcc80ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 --2- 172.21.15.33:0/1563814612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc80c7cc0 0x7fbcc8144470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 >> 172.21.15.33:0/1563814612 conn(0x7fbcc80c8ae0 msgr2=0x7fbcc8069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 shutdown_connections 2023-10-06T02:45:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:29.957+0000 7fbcce0cd700 1 -- 172.21.15.33:0/1563814612 wait complete. 2023-10-06T02:45:30.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:45:23.146455+0000', 'modified': '2023-10-06T02:45:28.315508+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2898450531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 2898450531}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2898450531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-10-06T02:45:30.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:45:30.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:45:30.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 Processor -- start 2023-10-06T02:45:30.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- start start 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4dfc0aec50 0x7f4dfc0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dfc1592e0 con 0x7f4dfc0684d0 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dfc0b5ae0 con 0x7f4dfc068950 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b2070 con 0x7f4dfc0aec50 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4df27fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4dfc0684d0 0x7f4dfc0b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60320/0 (socket says 172.21.15.33:60320) 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4df27fc700 1 -- 172.21.15.33:0/3954130639 learned_addr learned my addr 172.21.15.33:0/3954130639 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3954130639 >> v2:172.21.15.33:3300/0 conn(0x7f4dfc0aec50 0x7f4dfc0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfa7fc700 1 -- 172.21.15.33:0/3954130639 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443254671 0 0) 0x7f4dfc1592e0 con 0x7f4dfc0684d0 2023-10-06T02:45:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfa7fc700 1 -- 172.21.15.33:0/3954130639 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de8003570 con 0x7f4dfc0684d0 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3954130639 >> v1:172.21.15.167:6789/0 conn(0x7f4dfc068950 legacy=0x7f4dfc0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3954130639 >> v1:172.21.15.33:6789/0 conn(0x7f4dfc0684d0 legacy=0x7f4dfc0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3954130639 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0cedf0 con 0x7f4dfc0aec50 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3954130639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc0cc650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0098e0 tx=0x7f4dec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df10489fd005b71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfa7fc700 1 -- 172.21.15.33:0/3954130639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00b640 con 0x7f4dfc0aec50 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfa7fc700 1 -- 172.21.15.33:0/3954130639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00b7a0 con 0x7f4dfc0aec50 2023-10-06T02:45:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfa7fc700 1 -- 172.21.15.33:0/3954130639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00bb40 con 0x7f4dfc0aec50 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3954130639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0aec50 msgr2=0x7f4dfc0cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3954130639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc0cc650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0098e0 tx=0x7f4dec0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3954130639 shutdown_connections 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3954130639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc0cc650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.497+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3954130639 >> 172.21.15.33:0/3954130639 conn(0x7f4dfc067a40 msgr2=0x7f4dfc0b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.501+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3954130639 shutdown_connections 2023-10-06T02:45:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.501+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3954130639 wait complete. 2023-10-06T02:45:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.501+0000 7f4dfbfff700 1 Processor -- start 2023-10-06T02:45:30.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.501+0000 7f4dfbfff700 1 -- start start 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 0x7f4dfc159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4dfc068950 0x7f4dfc14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc14f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc14e040 con 0x7f4dfc0684d0 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc15a530 con 0x7f4dfc0aec50 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc153130 con 0x7f4dfc068950 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 0x7f4dfc159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 0x7f4dfc159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36502/0 (socket says 172.21.15.33:36502) 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3136241669 learned_addr learned my addr 172.21.15.33:0/3136241669 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc14f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4df27fc700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4dfc068950 0x7f4dfc14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4dfc068950 msgr2=0x7f4dfc14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4dfc068950 0x7f4dfc14f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dfc0aec50 msgr2=0x7f4dfc14f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc14f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3136241669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009420 con 0x7f4dfc0684d0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 0x7f4dfc159ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc05e8b0 tx=0x7f4dec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4df37fe700 1 -- 172.21.15.33:0/3136241669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec000d10 con 0x7f4dfc0684d0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4df37fe700 1 -- 172.21.15.33:0/3136241669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00bb20 con 0x7f4dfc0684d0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dfc1539d0 con 0x7f4dfc0684d0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4df37fe700 1 -- 172.21.15.33:0/3136241669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec011650 con 0x7f4dfc0684d0 2023-10-06T02:45:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.505+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc151340 con 0x7f4dfc0684d0 2023-10-06T02:45:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.509+0000 7f4df37fe700 1 -- 172.21.15.33:0/3136241669 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4dec0119c0 con 0x7f4dfc0684d0 2023-10-06T02:45:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.509+0000 7f4df37fe700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4ddc060580 0x7f4ddc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.509+0000 7f4df37fe700 1 -- 172.21.15.33:0/3136241669 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4dec07dde0 con 0x7f4dfc0684d0 2023-10-06T02:45:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.509+0000 7f4df27fc700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4ddc060580 0x7f4ddc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.509+0000 7f4df27fc700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4ddc060580 0x7f4ddc062a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0b1730 tx=0x7f4de4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.509+0000 7f4e03bb3740 1 Processor -- start 2023-10-06T02:45:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- start start 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 0x17e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156c710 0x17ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x1717f30 con 0x156cae0 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x1713ed0 con 0x156c710 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x16da0b0 con 0x15b7790 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 0x17e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 0x17e9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36510/0 (socket says 172.21.15.33:36510) 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 -- 172.21.15.33:0/830004708 learned_addr learned my addr 172.21.15.33:0/830004708 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4df27fc700 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156c710 0x17ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 -- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 msgr2=0x17ea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17ea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 -- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156c710 msgr2=0x17ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156c710 0x17ea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 -- 172.21.15.33:0/830004708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15b89e0 con 0x156cae0 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4df27fc700 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156c710 0x17ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 0x17e9e50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4dec051c50 tx=0x7f4dec07be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1809dd787f2f08a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4ddbfff700 1 -- 172.21.15.33:0/830004708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec067170 con 0x156cae0 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4ddbfff700 1 -- 172.21.15.33:0/830004708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec052400 con 0x156cae0 2023-10-06T02:45:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4ddbfff700 1 -- 172.21.15.33:0/830004708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec054f50 con 0x156cae0 2023-10-06T02:45:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- 172.21.15.33:0/830004708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 msgr2=0x17e9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 0x17e9e50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4dec051c50 tx=0x7f4dec07be10 comp rx=0 tx=0).stop 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- 172.21.15.33:0/830004708 shutdown_connections 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17ea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156c710 0x17ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/830004708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156cae0 0x17e9e50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- 172.21.15.33:0/830004708 >> 172.21.15.33:0/830004708 conn(0x156ceb0 msgr2=0x17a6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- 172.21.15.33:0/830004708 shutdown_connections 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.513+0000 7f4e03bb3740 1 -- 172.21.15.33:0/830004708 wait complete. 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 -- start start 2023-10-06T02:45:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 Processor -- start 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d9d50 con 0x156c710 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b2970 con 0x156cae0 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x16dab00 con 0x15b7790 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 10 client.0 _ll_register_callbacks cb 0x175db00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17a6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36526/0 (socket says 172.21.15.33:36526) 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3221388529 learned_addr learned my addr 172.21.15.33:0/3221388529 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4df27fc700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 msgr2=0x17d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 msgr2=0x17cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 -- 172.21.15.33:0/3221388529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc151340 con 0x156c710 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17a6740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4dec051c50 tx=0x7f4dec024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6067828455b6f149 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T02:45:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dd97fa700 1 -- 172.21.15.33:0/3221388529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec05c070 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dd97fa700 1 -- 172.21.15.33:0/3221388529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4dec054f50 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16b2cc0 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4dd97fa700 1 -- 172.21.15.33:0/3221388529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dec058d20 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 10 client.4443 fetch_fsmap learned FSMap version 7 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.517+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x16e2370 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4dd97fa700 1 -- 172.21.15.33:0/3221388529 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16b2cc0 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4dd97fa700 1 -- 172.21.15.33:0/3221388529 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 7) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f4dec02a350 con 0x156c710 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 10 client.4443 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 20 client.4443 populate_metadata read hostname 'smithi033' 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17f66b0 0x17f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 4 client.4443 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:45:30.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 --> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16b63e0 con 0x17f66b0 2023-10-06T02:45:30.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4df27fc700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17f66b0 0x17f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4df27fc700 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17f66b0 0x17f8a90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcff68c3d4746a4e server_cookie=bcea69778fbdb3a9 in_seq=0 out_seq=0 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4dd97fa700 10 client.4443 ms_handle_connect on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4dd97fa700 1 -- 172.21.15.33:0/3221388529 <== mds.0 v2:172.21.15.33:6834/2898450531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x16b63e0 con 0x17f66b0 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4dd97fa700 10 client.4443 handle_command_reply: tid=0 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 client.4443 shutdown 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 msgr2=0x17a6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17a6740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4dec051c50 tx=0x7f4dec024000 comp rx=0 tx=0).stop 2023-10-06T02:45:30.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 shutdown_connections 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x15b7790 0x17d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17f66b0 0x17f8a90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/3221388529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17a6740 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4dd97fa700 0 client.4443 ms_handle_reset on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 >> 172.21.15.33:0/3221388529 conn(0x156ceb0 msgr2=0x179eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 shutdown_connections 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 1 -- 172.21.15.33:0/3221388529 wait complete. 2023-10-06T02:45:30.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.521+0000 7f4e03bb3740 20 client.4443 trim_cache size 0 max 16384 2023-10-06T02:45:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 Processor -- start 2023-10-06T02:45:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 -- start start 2023-10-06T02:45:30.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156c710 0x17f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 0x17f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x16e8610 con 0x15b7790 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b2970 con 0x156cae0 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x16e7040 con 0x156c710 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 0x17f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 0x17f9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36538/0 (socket says 172.21.15.33:36538) 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 -- 172.21.15.33:0/1349220730 learned_addr learned my addr 172.21.15.33:0/1349220730 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4df27fc700 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156c710 0x17f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 -- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156c710 msgr2=0x17f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156c710 0x17f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 -- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 msgr2=0x17f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4dfb7fe700 1 -- 172.21.15.33:0/1349220730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16c8d80 con 0x15b7790 2023-10-06T02:45:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.537+0000 7f4df27fc700 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 0x17f9510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4df4002d20 tx=0x7f4df4001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3648508eb2526de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4df0ff9700 1 -- 172.21.15.33:0/1349220730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4013ad0 con 0x15b7790 2023-10-06T02:45:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4df0ff9700 1 -- 172.21.15.33:0/1349220730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df4013c30 con 0x15b7790 2023-10-06T02:45:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4df0ff9700 1 -- 172.21.15.33:0/1349220730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4016450 con 0x15b7790 2023-10-06T02:45:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1349220730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 msgr2=0x17f9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 0x17f9510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4df4002d20 tx=0x7f4df4001690 comp rx=0 tx=0).stop 2023-10-06T02:45:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1349220730 shutdown_connections 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x15b7790 0x17f9510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x156cae0 0x17f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1349220730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156c710 0x17f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1349220730 >> 172.21.15.33:0/1349220730 conn(0x156ceb0 msgr2=0x179f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1349220730 shutdown_connections 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1349220730 wait complete. 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 -- start start 2023-10-06T02:45:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.541+0000 7f4e03bb3740 1 Processor -- start 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156cae0 0x17cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x15b7790 0x17cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x1712d60 con 0x156c710 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x1543060 con 0x15b7790 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x156a430 con 0x156cae0 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 10 client.0 _ll_register_callbacks cb 0x16300e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:45:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36550/0 (socket says 172.21.15.33:36550) 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 -- 172.21.15.33:0/1343984016 learned_addr learned my addr 172.21.15.33:0/1343984016 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4df27fc700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156cae0 0x17cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfb7fe700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x15b7790 0x17cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 -- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156cae0 msgr2=0x17cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156cae0 0x17cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 -- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x15b7790 msgr2=0x17cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x15b7790 0x17cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 -- 172.21.15.33:0/1343984016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df400f040 con 0x156c710 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dfaffd700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17cab90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0519c0 tx=0x7f4dec078070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9047503e32af4b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 1 -- 172.21.15.33:0/1343984016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec05c040 con 0x156c710 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 1 -- 172.21.15.33:0/1343984016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4dec054f50 con 0x156c710 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dec052400 con 0x156c710 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 1 -- 172.21.15.33:0/1343984016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dec067170 con 0x156c710 2023-10-06T02:45:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 10 client.4449 fetch_fsmap learned FSMap version 7 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x16bd130 con 0x156c710 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 1 -- 172.21.15.33:0/1343984016 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec052400 con 0x156c710 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 1 -- 172.21.15.33:0/1343984016 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 7) v1 ==== 3108+0+0 (secure 0 0 0) 0x7f4dec0511b0 con 0x156c710 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 10 client.4449 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 10 client.4449 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 20 client.4449 populate_metadata read hostname 'smithi033' 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17cbfd0 0x184b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 4 client.4449 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 --> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] -- command(tid 0: {"prefix": "status"}) v1 -- 0x16e8410 con 0x17cbfd0 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4df27fc700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17cbfd0 0x184b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4dd97fa700 10 client.4449 ms_handle_connect on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.545+0000 7f4df27fc700 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17cbfd0 0x184b120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce3dfb8f55dc98b server_cookie=ec9af985b27f35ec in_seq=0 out_seq=0 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dd97fa700 1 -- 172.21.15.33:0/1343984016 <== mds.0 v2:172.21.15.33:6834/2898450531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x16e8410 con 0x17cbfd0 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dd97fa700 10 client.4449 handle_command_reply: tid=0 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 client.4449 shutdown 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 msgr2=0x17cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17cab90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0519c0 tx=0x7f4dec078070 comp rx=0 tx=0).stop 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 shutdown_connections 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x156cae0 0x17cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x17cbfd0 0x184b120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x15b7790 0x17cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 --2- 172.21.15.33:0/1343984016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x156c710 0x17cab90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dd97fa700 0 client.4449 ms_handle_reset on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 >> 172.21.15.33:0/1343984016 conn(0x156ceb0 msgr2=0x179f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 shutdown_connections 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 1 -- 172.21.15.33:0/1343984016 wait complete. 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4e03bb3740 20 client.4449 trim_cache size 0 max 16384 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8ff12af2-25a8-465d-8914-d29bca58b4dd", 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4287, 2023-10-06T02:45:30.557 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.7727182170000004, 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 7, 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 29, 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 29, 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 11.817760528999999 2023-10-06T02:45:30.558 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4ddc060580 msgr2=0x7f4ddc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4ddc060580 0x7f4ddc062a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0b1730 tx=0x7f4de4009040 comp rx=0 tx=0).stop 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 msgr2=0x7f4dfc159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 0x7f4dfc159ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc05e8b0 tx=0x7f4dec017000 comp rx=0 tx=0).stop 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 shutdown_connections 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4ddc060580 0x7f4ddc062a30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4dfc068950 0x7f4dfc14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dfc0aec50 0x7f4dfc14f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 --2- 172.21.15.33:0/3136241669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc0684d0 0x7f4dfc159ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 >> 172.21.15.33:0/3136241669 conn(0x7f4dfc067a40 msgr2=0x7f4dfc1587d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 shutdown_connections 2023-10-06T02:45:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.549+0000 7f4dfbfff700 1 -- 172.21.15.33:0/3136241669 wait complete. 2023-10-06T02:45:30.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:45:30.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f5261bb7700 1 Processor -- start 2023-10-06T02:45:30.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f5261bb7700 1 -- start start 2023-10-06T02:45:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f5261bb7700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f525c155350 0x7f525c0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f5261bb7700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0c86b0 con 0x7f525c06e540 2023-10-06T02:45:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f5261bb7700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0c7aa0 con 0x7f525c06e170 2023-10-06T02:45:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f5261bb7700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c068080 con 0x7f525c155350 2023-10-06T02:45:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f525c06e540 0x7f525c154c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60332/0 (socket says 172.21.15.33:60332) 2023-10-06T02:45:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525affd700 1 -- 172.21.15.33:0/2504763890 learned_addr learned my addr 172.21.15.33:0/2504763890 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525bfff700 1 --2- 172.21.15.33:0/2504763890 >> v2:172.21.15.167:3300/0 conn(0x7f525c155350 0x7f525c0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525a7fc700 1 -- 172.21.15.33:0/2504763890 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084245558 0 0) 0x7f525c0c86b0 con 0x7f525c06e540 2023-10-06T02:45:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525a7fc700 1 -- 172.21.15.33:0/2504763890 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5248004680 con 0x7f525c06e540 2023-10-06T02:45:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525bfff700 1 -- 172.21.15.33:0/2504763890 >> v1:172.21.15.167:6789/0 conn(0x7f525c06e170 legacy=0x7f525c067970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525bfff700 1 -- 172.21.15.33:0/2504763890 >> v1:172.21.15.33:6789/0 conn(0x7f525c06e540 legacy=0x7f525c154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525bfff700 1 -- 172.21.15.33:0/2504763890 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0c7d80 con 0x7f525c155350 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525bfff700 1 --2- 172.21.15.33:0/2504763890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c155350 0x7f525c0c7450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5254009e90 tx=0x7f5254009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac665966356102e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525a7fc700 1 -- 172.21.15.33:0/2504763890 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525400e070 con 0x7f525c155350 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525a7fc700 1 -- 172.21.15.33:0/2504763890 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525400bc40 con 0x7f525c155350 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.681+0000 7f525a7fc700 1 -- 172.21.15.33:0/2504763890 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254012430 con 0x7f525c155350 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- 172.21.15.33:0/2504763890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c155350 msgr2=0x7f525c0c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 --2- 172.21.15.33:0/2504763890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c155350 0x7f525c0c7450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5254009e90 tx=0x7f5254009780 comp rx=0 tx=0).stop 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- 172.21.15.33:0/2504763890 shutdown_connections 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 --2- 172.21.15.33:0/2504763890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c155350 0x7f525c0c7450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- 172.21.15.33:0/2504763890 >> 172.21.15.33:0/2504763890 conn(0x7f525c06aa50 msgr2=0x7f525c0cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- 172.21.15.33:0/2504763890 shutdown_connections 2023-10-06T02:45:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- 172.21.15.33:0/2504763890 wait complete. 2023-10-06T02:45:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 Processor -- start 2023-10-06T02:45:30.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- start start 2023-10-06T02:45:30.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525c06e170 0x7f525c158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c06e540 0x7f525c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 0x7f525c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0c4c20 con 0x7f525c155350 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0ae4b0 con 0x7f525c06e540 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f5261bb7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c159790 con 0x7f525c06e170 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f525bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 0x7f525c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f525bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 0x7f525c0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36564/0 (socket says 172.21.15.33:36564) 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.685+0000 7f525bfff700 1 -- 172.21.15.33:0/1748644871 learned_addr learned my addr 172.21.15.33:0/1748644871 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525affd700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c06e540 0x7f525c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525b7fe700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525c06e170 0x7f525c158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525bfff700 1 -- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525c06e170 msgr2=0x7f525c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525bfff700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525c06e170 0x7f525c158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525bfff700 1 -- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c06e540 msgr2=0x7f525c0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525bfff700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c06e540 0x7f525c0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525bfff700 1 -- 172.21.15.33:0/1748644871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254009a20 con 0x7f525c155350 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f525bfff700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 0x7f525c0adf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f525400ba00 tx=0x7f5254019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254024400 con 0x7f525c155350 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5254024580 con 0x7f525c155350 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c159a70 con 0x7f525c155350 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254024940 con 0x7f525c155350 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c0c5350 con 0x7f525c155350 2023-10-06T02:45:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.689+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c0c4ff0 con 0x7f525c155350 2023-10-06T02:45:30.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.693+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f525402d450 con 0x7f525c155350 2023-10-06T02:45:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.693+0000 7f52537fe700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f523c060550 0x7f523c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.693+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f525407ea80 con 0x7f525c155350 2023-10-06T02:45:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.693+0000 7f525b7fe700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f523c060550 0x7f523c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.693+0000 7f525b7fe700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f523c060550 0x7f523c062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5244000c00 tx=0x7f5244007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.701+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f525404ecd0 con 0x7f525c155350 2023-10-06T02:45:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.901+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f525c0710b0 con 0x7f525c155350 2023-10-06T02:45:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.901+0000 7f52537fe700 1 -- 172.21.15.33:0/1748644871 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f52540522f0 con 0x7f525c155350 2023-10-06T02:45:30.907 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T02:45:30.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f523c060550 msgr2=0x7f523c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f523c060550 0x7f523c062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5244000c00 tx=0x7f5244007ea0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 msgr2=0x7f525c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 0x7f525c0adf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f525400ba00 tx=0x7f5254019000 comp rx=0 tx=0).stop 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 shutdown_connections 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f525c155350 0x7f525c0adf70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f525c06e540 0x7f525c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f523c060550 0x7f523c062a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 --2- 172.21.15.33:0/1748644871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f525c06e170 0x7f525c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 >> 172.21.15.33:0/1748644871 conn(0x7f525c06aa50 msgr2=0x7f525c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 shutdown_connections 2023-10-06T02:45:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:30.905+0000 7f5261bb7700 1 -- 172.21.15.33:0/1748644871 wait complete. 2023-10-06T02:45:30.928 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:45:30.929 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T02:45:30.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:30.929 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T02:45:30.935 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:45:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42581sec preferred_lft 42581sec 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:72:46:44:8b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:45:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:30.938 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T02:45:30.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T02:45:30.938 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:30.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:30.986 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:45:30.997 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T02:45:30.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:30.999 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T02:45:31.027 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T02:45:31.027 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:45:31.027 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T02:45:31.027 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:45:31.027 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T02:45:31.027 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:45:31.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:31.028 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:31.028 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:31.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:45:31.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:45:31.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:45:31.029 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:31.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:31.070 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:45:31.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:31.077 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-10-06T02:45:31.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:31.127 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:31.127 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:31.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T02:45:31.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-06T02:45:31.127 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:31.161 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-06T02:45:31.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:31.161 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-06T02:45:31.162 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:31.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:31.239 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:31.240 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:31.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T02:45:31.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T02:45:31.240 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:31.269 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-06T02:45:31.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:31.269 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:31.276 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-06T02:45:31.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:31.277 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:31.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:31.337 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:31.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:31.351 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:31.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:31.352 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:31.358 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T02:45:31.358 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-06T02:45:31.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:31.419 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:31.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:31.430 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:31.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:31.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:31.432 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:31.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:2023-10-06T02:45:31.485+0000 7fee008c6380 -1 init, newargv = 0x5625d62fb9f0 newargc=15 2023-10-06T02:45:31.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[39884]: starting ceph client 2023-10-06T02:45:31.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[39884]: starting fuse 2023-10-06T02:45:32.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:32.556 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:32.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:32.568 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:32.569 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:32.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:32.570 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:32.623 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:45:32.623 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T02:45:32.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:45:32.623 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2023-10-06T02:45:32.623 DEBUG:teuthology.orchestra.run.smithi033:> import re 2023-10-06T02:45:32.623 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2023-10-06T02:45:32.624 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> return f 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.admin")) 2023-10-06T02:45:32.625 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:45:32.711 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.admin.39884.asok 2023-10-06T02:45:32.895 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.39884.asok 2023-10-06T02:45:32.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:32.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.39884.asok status 2023-10-06T02:45:33.002 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "admin", 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "39884", 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2023-10-06T02:45:33.003 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4461, 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4461 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2224040024 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:33.004 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2224040024 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4461 192.168.0.1:0/2224040024", 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/2224040024", 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 7, 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 29, 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2023-10-06T02:45:33.005 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:45:33.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:33.012 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2023-10-06T02:45:33.013 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:33.019 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2023-10-06T02:45:33.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:33.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:33.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:33.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:33.039 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-06T02:45:33.127 INFO:teuthology.orchestra.run.smithi033.stdout:# file: . 2023-10-06T02:45:33.127 INFO:teuthology.orchestra.run.smithi033.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-06T02:45:33.128 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:33.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T02:45:33.129 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:45:33.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-06T02:45:33.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[39884]: fuse finished with error 0 and tester_r 0 2023-10-06T02:45:33.156 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T02:45:39.258 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:45:39.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:39.258 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-06T02:45:39.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:39.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 Processor -- start 2023-10-06T02:45:39.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 -- start start 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe20c06c010 0x7fe20c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c0abb50 0x7fe20c069540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c15a390 con 0x7fe20c155380 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c159570 con 0x7fe20c06c010 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c069a80 con 0x7fe20c0abb50 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe20c06c010 0x7fe20c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe20c06c010 0x7fe20c06c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56532/0 (socket says 172.21.15.33:56532) 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 -- 172.21.15.33:0/3316173175 learned_addr learned my addr 172.21.15.33:0/3316173175 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:39.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20ad9d700 1 --1- 172.21.15.33:0/3316173175 >> v1:172.21.15.33:6789/0 conn(0x7fe20c155380 0x7fe20c155820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58152/0 (socket says 172.21.15.33:58152) 2023-10-06T02:45:39.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe209d9b700 1 --2- 172.21.15.33:0/3316173175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c0abb50 0x7fe20c069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 -- 172.21.15.33:0/3316173175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c0abb50 msgr2=0x7fe20c069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:39.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 --2- 172.21.15.33:0/3316173175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c0abb50 0x7fe20c069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 -- 172.21.15.33:0/3316173175 >> v1:172.21.15.33:6789/0 conn(0x7fe20c155380 legacy=0x7fe20c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 -- 172.21.15.33:0/3316173175 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c159780 con 0x7fe20c06c010 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20a59c700 1 --2- 172.21.15.33:0/3316173175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c06c010 0x7fe20c06c3e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009990 tx=0x7fe1fc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b004e9ed6536a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20959a700 1 -- 172.21.15.33:0/3316173175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc005c10 con 0x7fe20c06c010 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20959a700 1 -- 172.21.15.33:0/3316173175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc005d70 con 0x7fe20c06c010 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20959a700 1 -- 172.21.15.33:0/3316173175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00b880 con 0x7fe20c06c010 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 -- 172.21.15.33:0/3316173175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c06c010 msgr2=0x7fe20c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.385+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3316173175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c06c010 0x7fe20c06c3e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009990 tx=0x7fe1fc009220 comp rx=0 tx=0).stop 2023-10-06T02:45:39.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- 172.21.15.33:0/3316173175 shutdown_connections 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3316173175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c0abb50 0x7fe20c069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3316173175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c06c010 0x7fe20c06c3e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- 172.21.15.33:0/3316173175 >> 172.21.15.33:0/3316173175 conn(0x7fe20c06e6c0 msgr2=0x7fe20c06ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- 172.21.15.33:0/3316173175 shutdown_connections 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- 172.21.15.33:0/3316173175 wait complete. 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 Processor -- start 2023-10-06T02:45:39.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- start start 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20c06c010 0x7fe20c14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 0x7fe20c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c155380 0x7fe20c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1519e0 con 0x7fe20c0abb50 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c14c800 con 0x7fe20c06c010 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c150e60 con 0x7fe20c155380 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe209d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 0x7fe20c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe209d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 0x7fe20c14a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56540/0 (socket says 172.21.15.33:56540) 2023-10-06T02:45:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe209d9b700 1 -- 172.21.15.33:0/3493505518 learned_addr learned my addr 172.21.15.33:0/3493505518 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20a59c700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20c06c010 0x7fe20c14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.389+0000 7fe20ad9d700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c155380 0x7fe20c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe209d9b700 1 -- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c155380 msgr2=0x7fe20c0ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe209d9b700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c155380 0x7fe20c0ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe209d9b700 1 -- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20c06c010 msgr2=0x7fe20c14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe209d9b700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20c06c010 0x7fe20c14fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe209d9b700 1 -- 172.21.15.33:0/3493505518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009480 con 0x7fe20c0abb50 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe209d9b700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 0x7fe20c14a810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe20000d860 tx=0x7fe200008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000125f0 con 0x7fe20c0abb50 2023-10-06T02:45:39.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe200012750 con 0x7fe20c0abb50 2023-10-06T02:45:39.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200012af0 con 0x7fe20c0abb50 2023-10-06T02:45:39.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c151140 con 0x7fe20c0abb50 2023-10-06T02:45:39.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c14ad50 con 0x7fe20c0abb50 2023-10-06T02:45:39.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.393+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c071040 con 0x7fe20c0abb50 2023-10-06T02:45:39.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.397+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe200013480 con 0x7fe20c0abb50 2023-10-06T02:45:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.397+0000 7fe1fa7fc700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1ec060550 0x7fe1ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.397+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe20007cf60 con 0x7fe20c0abb50 2023-10-06T02:45:39.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.397+0000 7fe20a59c700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1ec060550 0x7fe1ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.397+0000 7fe20a59c700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1ec060550 0x7fe1ec062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009260 tx=0x7fe1fc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.405+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe200050b90 con 0x7fe20c0abb50 2023-10-06T02:45:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.565+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe20c159a60 con 0x7fe20c0abb50 2023-10-06T02:45:39.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.565+0000 7fe1fa7fc700 1 -- 172.21.15.33:0/3493505518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fe200050720 con 0x7fe20c0abb50 2023-10-06T02:45:39.574 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:39.574 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1ec060550 msgr2=0x7fe1ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1ec060550 0x7fe1ec062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009260 tx=0x7fe1fc016000 comp rx=0 tx=0).stop 2023-10-06T02:45:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 msgr2=0x7fe20c14a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 0x7fe20c14a810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe20000d860 tx=0x7fe200008040 comp rx=0 tx=0).stop 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 shutdown_connections 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1ec060550 0x7fe1ec062a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe20c155380 0x7fe20c0ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe20c0abb50 0x7fe20c14a810 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 --2- 172.21.15.33:0/3493505518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe20c06c010 0x7fe20c14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 >> 172.21.15.33:0/3493505518 conn(0x7fe20c06e6c0 msgr2=0x7fe20c06eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 shutdown_connections 2023-10-06T02:45:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.569+0000 7fe20b59e700 1 -- 172.21.15.33:0/3493505518 wait complete. 2023-10-06T02:45:39.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.689+0000 7fa93f59e700 1 Processor -- start 2023-10-06T02:45:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.689+0000 7fa93f59e700 1 -- start start 2023-10-06T02:45:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa940151190 0x7fa9400b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9400b1f10 0x7fa9400cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9400caae0 con 0x7fa940150dc0 2023-10-06T02:45:39.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94006cd80 con 0x7fa940151190 2023-10-06T02:45:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b2350 con 0x7fa9400b1f10 2023-10-06T02:45:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa940151190 0x7fa9400b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9400b1f10 0x7fa9400cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa940151190 0x7fa9400b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:46454/0 (socket says 172.21.15.33:46454) 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 -- 172.21.15.33:0/4031968947 learned_addr learned my addr 172.21.15.33:0/4031968947 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93e59c700 1 --1- 172.21.15.33:0/4031968947 >> v1:172.21.15.167:6789/0 conn(0x7fa940150dc0 0x7fa94006bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57496/0 (socket says 172.21.15.33:57496) 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 -- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9400b1f10 msgr2=0x7fa9400cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 --2- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9400b1f10 0x7fa9400cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 -- 172.21.15.33:0/4031968947 >> v1:172.21.15.167:6789/0 conn(0x7fa940150dc0 legacy=0x7fa94006bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 -- 172.21.15.33:0/4031968947 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94006c4f0 con 0x7fa940151190 2023-10-06T02:45:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93ed9d700 1 --2- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9400b1f10 0x7fa9400cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa937fff700 1 --2- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa9400b19d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9280099b0 tx=0x7fa928009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a985532c3a4b7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93dd9b700 1 -- 172.21.15.33:0/4031968947 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928009ea0 con 0x7fa940151190 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93dd9b700 1 -- 172.21.15.33:0/4031968947 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92800b860 con 0x7fa940151190 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93dd9b700 1 -- 172.21.15.33:0/4031968947 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800bc00 con 0x7fa940151190 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 msgr2=0x7fa9400b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 --2- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa9400b19d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa9280099b0 tx=0x7fa928009250 comp rx=0 tx=0).stop 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- 172.21.15.33:0/4031968947 shutdown_connections 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 --2- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9400b1f10 0x7fa9400cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 --2- 172.21.15.33:0/4031968947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa9400b19d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- 172.21.15.33:0/4031968947 >> 172.21.15.33:0/4031968947 conn(0x7fa94006e6c0 msgr2=0x7fa94006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- 172.21.15.33:0/4031968947 shutdown_connections 2023-10-06T02:45:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.693+0000 7fa93f59e700 1 -- 172.21.15.33:0/4031968947 wait complete. 2023-10-06T02:45:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 Processor -- start 2023-10-06T02:45:39.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 -- start start 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 0x7fa9400afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa940150dc0 0x7fa9400ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa94016c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b06e0 con 0x7fa9400b1f10 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b09c0 con 0x7fa940151190 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b0ca0 con 0x7fa940150dc0 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 0x7fa9400afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 0x7fa9400afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56544/0 (socket says 172.21.15.33:56544) 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 -- 172.21.15.33:0/2075854989 learned_addr learned my addr 172.21.15.33:0/2075854989 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa937fff700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa940150dc0 0x7fa9400ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93ed9d700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa94016c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 -- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa940150dc0 msgr2=0x7fa9400ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa940150dc0 0x7fa9400ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 -- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 msgr2=0x7fa94016c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa94016c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 -- 172.21.15.33:0/2075854989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa938009a20 con 0x7fa9400b1f10 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.697+0000 7fa93e59c700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 0x7fa9400afe50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c590 tx=0x7fa930001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930005ac0 con 0x7fa9400b1f10 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930005c20 con 0x7fa9400b1f10 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9280094f0 con 0x7fa9400b1f10 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300070d0 con 0x7fa9400b1f10 2023-10-06T02:45:39.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9400b0390 con 0x7fa9400b1f10 2023-10-06T02:45:39.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9400b11a0 con 0x7fa9400b1f10 2023-10-06T02:45:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.701+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa930007440 con 0x7fa9400b1f10 2023-10-06T02:45:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.705+0000 7fa9367fc700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa920060550 0x7fa920062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.705+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa93007bff0 con 0x7fa9400b1f10 2023-10-06T02:45:39.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.705+0000 7fa937fff700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa920060550 0x7fa920062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:39.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.705+0000 7fa937fff700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa920060550 0x7fa920062a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9280099b0 tx=0x7fa928017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.713+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa93004c210 con 0x7fa9400b1f10 2023-10-06T02:45:39.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.877+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa940071040 con 0x7fa9400b1f10 2023-10-06T02:45:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa9367fc700 1 -- 172.21.15.33:0/2075854989 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7fa93004f830 con 0x7fa9400b1f10 2023-10-06T02:45:39.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:39.885 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa920060550 msgr2=0x7fa920062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa920060550 0x7fa920062a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9280099b0 tx=0x7fa928017000 comp rx=0 tx=0).stop 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 msgr2=0x7fa9400afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 0x7fa9400afe50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c590 tx=0x7fa930001070 comp rx=0 tx=0).stop 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 shutdown_connections 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa920060550 0x7fa920062a00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa940150dc0 0x7fa9400ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa940151190 0x7fa94016c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 --2- 172.21.15.33:0/2075854989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9400b1f10 0x7fa9400afe50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:39.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 >> 172.21.15.33:0/2075854989 conn(0x7fa94006e6c0 msgr2=0x7fa94006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:39.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 shutdown_connections 2023-10-06T02:45:39.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:39.881+0000 7fa93f59e700 1 -- 172.21.15.33:0/2075854989 wait complete. 2023-10-06T02:45:39.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:45:40.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe598be6700 1 Processor -- start 2023-10-06T02:45:40.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe598be6700 1 -- start start 2023-10-06T02:45:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe598be6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5940cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe598be6700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940cc640 con 0x7fe594156500 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe598be6700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940cc920 con 0x7fe59406c170 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe598be6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940ccc00 con 0x7fe5940cda70 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe593fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe594156500 0x7fe594156980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58164/0 (socket says 172.21.15.33:58164) 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe593fff700 1 -- 172.21.15.33:0/2726985877 learned_addr learned my addr 172.21.15.33:0/2726985877 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5937fe700 1 --2- 172.21.15.33:0/2726985877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5940cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5927fc700 1 -- 172.21.15.33:0/2726985877 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457468203 0 0) 0x7fe5940cc640 con 0x7fe594156500 2023-10-06T02:45:40.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5927fc700 1 -- 172.21.15.33:0/2726985877 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe57c003570 con 0x7fe594156500 2023-10-06T02:45:40.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5927fc700 1 -- 172.21.15.33:0/2726985877 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733529655 0 0) 0x7fe5940cc920 con 0x7fe59406c170 2023-10-06T02:45:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5937fe700 1 -- 172.21.15.33:0/2726985877 >> v1:172.21.15.167:6789/0 conn(0x7fe59406c170 legacy=0x7fe5940cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5937fe700 1 -- 172.21.15.33:0/2726985877 >> v1:172.21.15.33:6789/0 conn(0x7fe594156500 legacy=0x7fe594156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.005+0000 7fe5937fe700 1 -- 172.21.15.33:0/2726985877 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940ccee0 con 0x7fe5940cda70 2023-10-06T02:45:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe5937fe700 1 --2- 172.21.15.33:0/2726985877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5940cde40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009e90 tx=0x7fe58c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a279afa3bc10239c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe5927fc700 1 -- 172.21.15.33:0/2726985877 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00e070 con 0x7fe5940cda70 2023-10-06T02:45:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe5927fc700 1 -- 172.21.15.33:0/2726985877 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c00bc30 con 0x7fe5940cda70 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe5927fc700 1 -- 172.21.15.33:0/2726985877 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c012430 con 0x7fe5940cda70 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 -- 172.21.15.33:0/2726985877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 msgr2=0x7fe5940cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 --2- 172.21.15.33:0/2726985877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5940cde40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009e90 tx=0x7fe58c009780 comp rx=0 tx=0).stop 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 -- 172.21.15.33:0/2726985877 shutdown_connections 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 --2- 172.21.15.33:0/2726985877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5940cde40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 -- 172.21.15.33:0/2726985877 >> 172.21.15.33:0/2726985877 conn(0x7fe59406ad60 msgr2=0x7fe59406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:40.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 -- 172.21.15.33:0/2726985877 shutdown_connections 2023-10-06T02:45:40.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.009+0000 7fe598be6700 1 -- 172.21.15.33:0/2726985877 wait complete. 2023-10-06T02:45:40.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 Processor -- start 2023-10-06T02:45:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 -- start start 2023-10-06T02:45:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59406c170 0x7fe5941573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5941591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 0x7fe59414c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe594159730 con 0x7fe594156500 2023-10-06T02:45:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe594158690 con 0x7fe59406c170 2023-10-06T02:45:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe594157a80 con 0x7fe5940cda70 2023-10-06T02:45:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 0x7fe59414c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 0x7fe59414c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56558/0 (socket says 172.21.15.33:56558) 2023-10-06T02:45:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 -- 172.21.15.33:0/4032567170 learned_addr learned my addr 172.21.15.33:0/4032567170 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe5937fe700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59406c170 0x7fe5941573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe592ffd700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5941591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 -- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 msgr2=0x7fe5941591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5941591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 -- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59406c170 msgr2=0x7fe5941573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59406c170 0x7fe5941573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 -- 172.21.15.33:0/4032567170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c009a20 con 0x7fe594156500 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe593fff700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 0x7fe59414c920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c560 tx=0x7fe584007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584005b30 con 0x7fe594156500 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584005cd0 con 0x7fe594156500 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584007320 con 0x7fe594156500 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe594157d60 con 0x7fe594156500 2023-10-06T02:45:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.013+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe594158320 con 0x7fe594156500 2023-10-06T02:45:40.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.017+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5940c9a20 con 0x7fe594156500 2023-10-06T02:45:40.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.017+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe584007690 con 0x7fe594156500 2023-10-06T02:45:40.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.017+0000 7fe57b7fe700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe580060550 0x7fe580062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.017+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe58407bd80 con 0x7fe594156500 2023-10-06T02:45:40.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.017+0000 7fe5937fe700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe580060550 0x7fe580062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.017+0000 7fe5937fe700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe580060550 0x7fe580062a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0097c0 tx=0x7fe58c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.025+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe584049be0 con 0x7fe594156500 2023-10-06T02:45:40.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.225+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe594071040 con 0x7fe594156500 2023-10-06T02:45:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.377+0000 7fe57b7fe700 1 -- 172.21.15.33:0/4032567170 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe584017020 con 0x7fe594156500 2023-10-06T02:45:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe580060550 msgr2=0x7fe580062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe580060550 0x7fe580062a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0097c0 tx=0x7fe58c019000 comp rx=0 tx=0).stop 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 msgr2=0x7fe59414c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 0x7fe59414c920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c560 tx=0x7fe584007d50 comp rx=0 tx=0).stop 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 shutdown_connections 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe580060550 0x7fe580062a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5940cda70 0x7fe5941591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe594156500 0x7fe59414c920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 --2- 172.21.15.33:0/4032567170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe59406c170 0x7fe5941573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 >> 172.21.15.33:0/4032567170 conn(0x7fe59406ad60 msgr2=0x7fe5940ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 shutdown_connections 2023-10-06T02:45:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.381+0000 7fe598be6700 1 -- 172.21.15.33:0/4032567170 wait complete. 2023-10-06T02:45:40.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:45:40.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.513+0000 7f79b55c7700 1 Processor -- start 2023-10-06T02:45:40.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.513+0000 7f79b55c7700 1 -- start start 2023-10-06T02:45:40.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79b006e610 0x7f79b00c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00cad70 con 0x7f79b00c7ef0 2023-10-06T02:45:40.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00ca160 con 0x7f79b006e610 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00ca440 con 0x7f79b0155360 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79b006e610 0x7f79b00c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79b006e610 0x7f79b00c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56560/0 (socket says 172.21.15.33:56560) 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 -- 172.21.15.33:0/850042961 learned_addr learned my addr 172.21.15.33:0/850042961 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79aeffd700 1 --2- 172.21.15.33:0/850042961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 -- 172.21.15.33:0/850042961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 msgr2=0x7f79b00cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 --2- 172.21.15.33:0/850042961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b00cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 -- 172.21.15.33:0/850042961 >> v1:172.21.15.167:6789/0 conn(0x7f79b00c7ef0 legacy=0x7f79b006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 -- 172.21.15.33:0/850042961 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00ca720 con 0x7f79b006e610 2023-10-06T02:45:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79af7fe700 1 --2- 172.21.15.33:0/850042961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b00c8340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f20 tx=0x7f79a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916d344ceb56a46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:40.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79adffb700 1 -- 172.21.15.33:0/850042961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800e070 con 0x7f79b006e610 2023-10-06T02:45:40.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79adffb700 1 -- 172.21.15.33:0/850042961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800ba80 con 0x7f79b006e610 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79adffb700 1 -- 172.21.15.33:0/850042961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80092e0 con 0x7f79b006e610 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- 172.21.15.33:0/850042961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 msgr2=0x7f79b00c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 --2- 172.21.15.33:0/850042961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b00c8340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f20 tx=0x7f79a8009840 comp rx=0 tx=0).stop 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- 172.21.15.33:0/850042961 shutdown_connections 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 --2- 172.21.15.33:0/850042961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b00c8340 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 --2- 172.21.15.33:0/850042961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b00cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- 172.21.15.33:0/850042961 >> 172.21.15.33:0/850042961 conn(0x7f79b006a9e0 msgr2=0x7f79b00cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- 172.21.15.33:0/850042961 shutdown_connections 2023-10-06T02:45:40.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.517+0000 7f79b55c7700 1 -- 172.21.15.33:0/850042961 wait complete. 2023-10-06T02:45:40.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 Processor -- start 2023-10-06T02:45:40.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 -- start start 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b0166c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b00c7ef0 0x7f79b01403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b0169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0167140 con 0x7f79b006e610 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01418d0 con 0x7f79b00c7ef0 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79b55c7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0140cc0 con 0x7f79b0155360 2023-10-06T02:45:40.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b0166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b0166c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56574/0 (socket says 172.21.15.33:56574) 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79aeffd700 1 -- 172.21.15.33:0/3967584608 learned_addr learned my addr 172.21.15.33:0/3967584608 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.521+0000 7f79af7fe700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b0169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79ae7fc700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b00c7ef0 0x7f79b01403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79aeffd700 1 -- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 msgr2=0x7f79b0169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79aeffd700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b0169840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79aeffd700 1 -- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b00c7ef0 msgr2=0x7f79b01403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79aeffd700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b00c7ef0 0x7f79b01403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79aeffd700 1 -- 172.21.15.33:0/3967584608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009ae0 con 0x7f79b006e610 2023-10-06T02:45:40.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79aeffd700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b0166c00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79a000d990 tx=0x7f79a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:40.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0011670 con 0x7f79b006e610 2023-10-06T02:45:40.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00117d0 con 0x7f79b006e610 2023-10-06T02:45:40.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b0140fa0 con 0x7f79b006e610 2023-10-06T02:45:40.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0011b90 con 0x7f79b006e610 2023-10-06T02:45:40.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0141560 con 0x7f79b006e610 2023-10-06T02:45:40.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b00710b0 con 0x7f79b006e610 2023-10-06T02:45:40.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.525+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f79a000f410 con 0x7f79b006e610 2023-10-06T02:45:40.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.529+0000 7f7996ffd700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f799c060550 0x7f799c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:40.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.529+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f79a007cd20 con 0x7f79b006e610 2023-10-06T02:45:40.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.529+0000 7f79ae7fc700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f799c060550 0x7f799c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.529+0000 7f79ae7fc700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f799c060550 0x7f799c062a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79b00c6780 tx=0x7f79a4007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.537+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f79a004cec0 con 0x7f79b006e610 2023-10-06T02:45:40.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:40.733+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f79b00caa00 con 0x7f79b006e610 2023-10-06T02:45:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.385+0000 7f7996ffd700 1 -- 172.21.15.33:0/3967584608 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f79a00504e0 con 0x7f79b006e610 2023-10-06T02:45:41.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.389+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f799c060550 msgr2=0x7f799c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.389+0000 7f79b55c7700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f799c060550 0x7f799c062a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79b00c6780 tx=0x7f79a4007ec0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.389+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 msgr2=0x7f79b0166c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.389+0000 7f79b55c7700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b0166c00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79a000d990 tx=0x7f79a0008040 comp rx=0 tx=0).stop 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.389+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 shutdown_connections 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f799c060550 0x7f799c062a00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79b0155360 0x7f79b0169840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79b00c7ef0 0x7f79b01403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 --2- 172.21.15.33:0/3967584608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79b006e610 0x7f79b0166c00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 >> 172.21.15.33:0/3967584608 conn(0x7f79b006a9e0 msgr2=0x7f79b00b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 shutdown_connections 2023-10-06T02:45:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.393+0000 7f79b55c7700 1 -- 172.21.15.33:0/3967584608 wait complete. 2023-10-06T02:45:41.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 Processor -- start 2023-10-06T02:45:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 -- start start 2023-10-06T02:45:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87b80aec50 0x7f87b80cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 0x7f87b80b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b81592e0 con 0x7f87b80684d0 2023-10-06T02:45:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b80b2070 con 0x7f87b80aec50 2023-10-06T02:45:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bd5b8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b5ae0 con 0x7f87b8068950 2023-10-06T02:45:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87b7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87b80aec50 0x7f87b80cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bcdb7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 0x7f87b80b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87b7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87b80aec50 0x7f87b80cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:46486/0 (socket says 172.21.15.33:46486) 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87bcdb7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 0x7f87b80b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:49322/0 (socket says 172.21.15.33:49322) 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87af7fe700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f87b80684d0 0x7f87b80b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57524/0 (socket says 172.21.15.33:57524) 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.529+0000 7f87b7fff700 1 -- 172.21.15.33:0/867628724 learned_addr learned my addr 172.21.15.33:0/867628724 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bcdb7700 1 -- 172.21.15.33:0/867628724 >> v2:172.21.15.167:3300/0 conn(0x7f87b80aec50 msgr2=0x7f87b80cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bcdb7700 1 --2- 172.21.15.33:0/867628724 >> v2:172.21.15.167:3300/0 conn(0x7f87b80aec50 0x7f87b80cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bcdb7700 1 -- 172.21.15.33:0/867628724 >> v1:172.21.15.167:6789/0 conn(0x7f87b80684d0 legacy=0x7f87b80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bcdb7700 1 -- 172.21.15.33:0/867628724 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80cedf0 con 0x7f87b8068950 2023-10-06T02:45:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bcdb7700 1 --2- 172.21.15.33:0/867628724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 0x7f87b80b3220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009e60 tx=0x7f87b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af4e8483da8ac2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:41.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87b77fe700 1 -- 172.21.15.33:0/867628724 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000e070 con 0x7f87b8068950 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87b77fe700 1 -- 172.21.15.33:0/867628724 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b000bb60 con 0x7f87b8068950 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87b77fe700 1 -- 172.21.15.33:0/867628724 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0012430 con 0x7f87b8068950 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 -- 172.21.15.33:0/867628724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 msgr2=0x7f87b80b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/867628724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 0x7f87b80b3220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009e60 tx=0x7f87b0009780 comp rx=0 tx=0).stop 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 -- 172.21.15.33:0/867628724 shutdown_connections 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/867628724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b8068950 0x7f87b80b3220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/867628724 >> v2:172.21.15.167:3300/0 conn(0x7f87b80aec50 0x7f87b80cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 -- 172.21.15.33:0/867628724 >> 172.21.15.33:0/867628724 conn(0x7f87b8067a40 msgr2=0x7f87b80b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 -- 172.21.15.33:0/867628724 shutdown_connections 2023-10-06T02:45:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 -- 172.21.15.33:0/867628724 wait complete. 2023-10-06T02:45:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 Processor -- start 2023-10-06T02:45:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.533+0000 7f87bd5b8700 1 -- start start 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b80684d0 0x7f87b80cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 0x7f87b815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b80aec50 0x7f87b815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8153d30 con 0x7f87b8068950 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8159f70 con 0x7f87b80684d0 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b815a250 con 0x7f87b80aec50 2023-10-06T02:45:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 0x7f87b815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 0x7f87b815c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56586/0 (socket says 172.21.15.33:56586) 2023-10-06T02:45:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 -- 172.21.15.33:0/4220258603 learned_addr learned my addr 172.21.15.33:0/4220258603 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87b7fff700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b80684d0 0x7f87b80cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bcdb7700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b80aec50 0x7f87b815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 -- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b80aec50 msgr2=0x7f87b815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b80aec50 0x7f87b815cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 -- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b80684d0 msgr2=0x7f87b80cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b80684d0 0x7f87b80cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 -- 172.21.15.33:0/4220258603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009a20 con 0x7f87b8068950 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87af7fe700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 0x7f87b815c7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c7c0 tx=0x7f87a0007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0005a70 con 0x7f87b8068950 2023-10-06T02:45:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a0005bd0 con 0x7f87b8068950 2023-10-06T02:45:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b815a530 con 0x7f87b8068950 2023-10-06T02:45:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00070d0 con 0x7f87b8068950 2023-10-06T02:45:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b80cb6d0 con 0x7f87b8068950 2023-10-06T02:45:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.537+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8069550 con 0x7f87b8068950 2023-10-06T02:45:41.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.541+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f87a00074c0 con 0x7f87b8068950 2023-10-06T02:45:41.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.541+0000 7f87b4ff9700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8798060550 0x7f8798062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.541+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f87a007bff0 con 0x7f87b8068950 2023-10-06T02:45:41.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.541+0000 7f87b7fff700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8798060550 0x7f8798062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.541+0000 7f87b7fff700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8798060550 0x7f8798062a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006fd0 tx=0x7f87a8009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:41.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.545+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f87b8069550 con 0x7f87b8068950 2023-10-06T02:45:41.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.753+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87b8069550 con 0x7f87b8068950 2023-10-06T02:45:41.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.753+0000 7f87b4ff9700 1 -- 172.21.15.33:0/4220258603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f87a004c150 con 0x7f87b8068950 2023-10-06T02:45:41.758 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:41.758 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:45:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8798060550 msgr2=0x7f8798062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8798060550 0x7f8798062a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006fd0 tx=0x7f87a8009250 comp rx=0 tx=0).stop 2023-10-06T02:45:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 msgr2=0x7f87b815c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 0x7f87b815c7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c7c0 tx=0x7f87a0007f00 comp rx=0 tx=0).stop 2023-10-06T02:45:41.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 shutdown_connections 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8798060550 0x7f8798062a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87b80aec50 0x7f87b815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8068950 0x7f87b815c7f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 --2- 172.21.15.33:0/4220258603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b80684d0 0x7f87b80cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 >> 172.21.15.33:0/4220258603 conn(0x7f87b8067a40 msgr2=0x7f87b80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 shutdown_connections 2023-10-06T02:45:41.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.757+0000 7f87bd5b8700 1 -- 172.21.15.33:0/4220258603 wait complete. 2023-10-06T02:45:41.763 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:45:41.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:41.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 Processor -- start 2023-10-06T02:45:41.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 -- start start 2023-10-06T02:45:41.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6ac0aec50 0x7ff6ac0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0684d0 0x7ff6ac068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac068e60 con 0x7ff6ac0b2b10 2023-10-06T02:45:41.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac0b2070 con 0x7ff6ac0aec50 2023-10-06T02:45:41.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6abfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1592e0 con 0x7ff6ac0684d0 2023-10-06T02:45:41.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6ac0aec50 0x7ff6ac0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6ac0aec50 0x7ff6ac0cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56596/0 (socket says 172.21.15.33:56596) 2023-10-06T02:45:41.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1928814102 learned_addr learned my addr 172.21.15.33:0/1928814102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:41.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aa7fc700 1 --2- 172.21.15.33:0/1928814102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0684d0 0x7ff6ac068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1928814102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0684d0 msgr2=0x7ff6ac068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:41.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 --2- 172.21.15.33:0/1928814102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0684d0 0x7ff6ac068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1928814102 >> v1:172.21.15.167:6789/0 conn(0x7ff6ac0b2b10 legacy=0x7ff6ac0b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1928814102 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0b5ae0 con 0x7ff6ac0aec50 2023-10-06T02:45:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6aaffd700 1 --2- 172.21.15.33:0/1928814102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac0cc650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0099b0 tx=0x7ff69c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c2bbf4107bee58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6a9ffb700 1 -- 172.21.15.33:0/1928814102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c009ea0 con 0x7ff6ac0aec50 2023-10-06T02:45:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6a9ffb700 1 -- 172.21.15.33:0/1928814102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00b860 con 0x7ff6ac0aec50 2023-10-06T02:45:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.877+0000 7ff6a9ffb700 1 -- 172.21.15.33:0/1928814102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00bc00 con 0x7ff6ac0aec50 2023-10-06T02:45:41.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- 172.21.15.33:0/1928814102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0aec50 msgr2=0x7ff6ac0cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1928814102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac0cc650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0099b0 tx=0x7ff69c009250 comp rx=0 tx=0).stop 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- 172.21.15.33:0/1928814102 shutdown_connections 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1928814102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0684d0 0x7ff6ac068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1928814102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac0cc650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- 172.21.15.33:0/1928814102 >> 172.21.15.33:0/1928814102 conn(0x7ff6ac067a40 msgr2=0x7ff6ac0b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- 172.21.15.33:0/1928814102 shutdown_connections 2023-10-06T02:45:41.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- 172.21.15.33:0/1928814102 wait complete. 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 Processor -- start 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- start start 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 0x7ff6ac0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac1420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0b2b10 0x7ff6ac142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cbd50 con 0x7ff6ac0684d0 2023-10-06T02:45:41.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac143850 con 0x7ff6ac0aec50 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6abfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac142c40 con 0x7ff6ac0b2b10 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 0x7ff6ac0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 0x7ff6ac0cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56608/0 (socket says 172.21.15.33:56608) 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1041149552 learned_addr learned my addr 172.21.15.33:0/1041149552 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aa7fc700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac1420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6ab7fe700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0b2b10 0x7ff6ac142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0b2b10 msgr2=0x7ff6ac142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0b2b10 0x7ff6ac142610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac0aec50 msgr2=0x7ff6ac1420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac1420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.881+0000 7ff6aaffd700 1 -- 172.21.15.33:0/1041149552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0094f0 con 0x7ff6ac0684d0 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6aaffd700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 0x7ff6ac0cb810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac05e8b0 tx=0x7ff69c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c000d10 con 0x7ff6ac0684d0 2023-10-06T02:45:41.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00bbe0 con 0x7ff6ac0684d0 2023-10-06T02:45:41.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac142f20 con 0x7ff6ac0684d0 2023-10-06T02:45:41.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011670 con 0x7ff6ac0684d0 2023-10-06T02:45:41.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1434e0 con 0x7ff6ac0684d0 2023-10-06T02:45:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac071040 con 0x7ff6ac0684d0 2023-10-06T02:45:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff69c011aa0 con 0x7ff6ac0684d0 2023-10-06T02:45:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6a2ffd700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff68c060550 0x7ff68c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.885+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff69c07e050 con 0x7ff6ac0684d0 2023-10-06T02:45:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.889+0000 7ff6aa7fc700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff68c060550 0x7ff68c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.889+0000 7ff6aa7fc700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff68c060550 0x7ff68c062a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff694009fd0 tx=0x7ff69400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:41.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:41.893+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff69c04e270 con 0x7ff6ac0684d0 2023-10-06T02:45:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.057+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6ac068c40 con 0x7ff6ac0684d0 2023-10-06T02:45:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.061+0000 7ff6a2ffd700 1 -- 172.21.15.33:0/1041149552 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7ff69c051890 con 0x7ff6ac0684d0 2023-10-06T02:45:42.067 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:42.067 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff68c060550 msgr2=0x7ff68c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff68c060550 0x7ff68c062a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff694009fd0 tx=0x7ff69400b040 comp rx=0 tx=0).stop 2023-10-06T02:45:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 msgr2=0x7ff6ac0cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 0x7ff6ac0cb810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac05e8b0 tx=0x7ff69c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 shutdown_connections 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff68c060550 0x7ff68c062a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ac0b2b10 0x7ff6ac142610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ac0aec50 0x7ff6ac1420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 --2- 172.21.15.33:0/1041149552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ac0684d0 0x7ff6ac0cb810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 >> 172.21.15.33:0/1041149552 conn(0x7ff6ac067a40 msgr2=0x7ff6ac0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 shutdown_connections 2023-10-06T02:45:42.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.065+0000 7ff6abfff700 1 -- 172.21.15.33:0/1041149552 wait complete. 2023-10-06T02:45:42.081 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:45:42.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:45:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 Processor -- start 2023-10-06T02:45:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 -- start start 2023-10-06T02:45:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbbcc06a210 0x7fbbcc068270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbbcc15d360 0x7fbbcc0c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc068900 con 0x7fbbcc0c7f60 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc159310 con 0x7fbbcc15d360 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcbfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc0c9920 con 0x7fbbcc06a210 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcaffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbbcc0c7f60 0x7fbbcc069b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58168/0 (socket says 172.21.15.33:58168) 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcaffd700 1 -- 172.21.15.33:0/2281010754 learned_addr learned my addr 172.21.15.33:0/2281010754 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcb7fe700 1 --2- 172.21.15.33:0/2281010754 >> v2:172.21.15.33:3300/0 conn(0x7fbbcc15d360 0x7fbbcc0c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbca7fc700 1 --2- 172.21.15.33:0/2281010754 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc06a210 0x7fbbcc068270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbc9ffb700 1 -- 172.21.15.33:0/2281010754 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409883921 0 0) 0x7fbbcc068900 con 0x7fbbcc0c7f60 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbc9ffb700 1 -- 172.21.15.33:0/2281010754 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb8003570 con 0x7fbbcc0c7f60 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcb7fe700 1 -- 172.21.15.33:0/2281010754 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc06a210 msgr2=0x7fbbcc068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcb7fe700 1 --2- 172.21.15.33:0/2281010754 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc06a210 0x7fbbcc068270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcb7fe700 1 -- 172.21.15.33:0/2281010754 >> v1:172.21.15.33:6789/0 conn(0x7fbbcc0c7f60 legacy=0x7fbbcc069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.185+0000 7fbbcb7fe700 1 -- 172.21.15.33:0/2281010754 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0ca330 con 0x7fbbcc15d360 2023-10-06T02:45:42.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcb7fe700 1 --2- 172.21.15.33:0/2281010754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc15d360 0x7fbbcc0c64b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc06f780 tx=0x7fbbc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959bd8417eb86c70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbc9ffb700 1 -- 172.21.15.33:0/2281010754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4011630 con 0x7fbbcc15d360 2023-10-06T02:45:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbc9ffb700 1 -- 172.21.15.33:0/2281010754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc40117b0 con 0x7fbbcc15d360 2023-10-06T02:45:42.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbc9ffb700 1 -- 172.21.15.33:0/2281010754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4011b70 con 0x7fbbcc15d360 2023-10-06T02:45:42.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2281010754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc15d360 msgr2=0x7fbbcc0c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2281010754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc15d360 0x7fbbcc0c64b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc06f780 tx=0x7fbbc40091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2281010754 shutdown_connections 2023-10-06T02:45:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2281010754 >> v2:172.21.15.167:3300/0 conn(0x7fbbcc06a210 0x7fbbcc068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2281010754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc15d360 0x7fbbcc0c64b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2281010754 >> 172.21.15.33:0/2281010754 conn(0x7fbbcc0cc6b0 msgr2=0x7fbbcc0c8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2281010754 shutdown_connections 2023-10-06T02:45:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2281010754 wait complete. 2023-10-06T02:45:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 Processor -- start 2023-10-06T02:45:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- start start 2023-10-06T02:45:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 0x7fbbcc147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc0c7f60 0x7fbbcc0c1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc15d360 0x7fbbcc0c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc0c3e30 con 0x7fbbcc06a210 2023-10-06T02:45:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc0c43b0 con 0x7fbbcc0c7f60 2023-10-06T02:45:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc0b19a0 con 0x7fbbcc15d360 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.189+0000 7fbbcaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 0x7fbbcc147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 0x7fbbcc147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56620/0 (socket says 172.21.15.33:56620) 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbca7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc0c7f60 0x7fbbcc0c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 -- 172.21.15.33:0/2995411411 learned_addr learned my addr 172.21.15.33:0/2995411411 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcb7fe700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc15d360 0x7fbbcc0c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 -- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc15d360 msgr2=0x7fbbcc0c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc15d360 0x7fbbcc0c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 -- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc0c7f60 msgr2=0x7fbbcc0c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc0c7f60 0x7fbbcc0c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 -- 172.21.15.33:0/2995411411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4009420 con 0x7fbbcc06a210 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcaffd700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 0x7fbbcc147530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c590 tx=0x7fbbb4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4005b30 con 0x7fbbcc06a210 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4005cb0 con 0x7fbbcc06a210 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc0c2430 con 0x7fbbcc06a210 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4007320 con 0x7fbbcc06a210 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc0c4040 con 0x7fbbcc06a210 2023-10-06T02:45:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc071040 con 0x7fbbcc06a210 2023-10-06T02:45:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.193+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbbb401b3b0 con 0x7fbbcc06a210 2023-10-06T02:45:42.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.197+0000 7fbbc2ffd700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbbac060550 0x7fbbac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.197+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbbb407c0e0 con 0x7fbbcc06a210 2023-10-06T02:45:42.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.197+0000 7fbbca7fc700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbbac060550 0x7fbbac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.197+0000 7fbbca7fc700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbbac060550 0x7fbbac062a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.201+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbbb404c280 con 0x7fbbcc06a210 2023-10-06T02:45:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.357+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbcc0ca610 con 0x7fbbac060550 2023-10-06T02:45:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.381+0000 7fbbc2ffd700 1 -- 172.21.15.33:0/2995411411 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367276 (secure 0 0 0) 0x7fbbcc0ca610 con 0x7fbbac060550 2023-10-06T02:45:42.390 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbbac060550 msgr2=0x7fbbac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbbac060550 0x7fbbac062a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc00b040 comp rx=0 tx=0).stop 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 msgr2=0x7fbbcc147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 0x7fbbcc147530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c590 tx=0x7fbbb4007e00 comp rx=0 tx=0).stop 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 shutdown_connections 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbbac060550 0x7fbbac062a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbcc15d360 0x7fbbcc0c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbcc0c7f60 0x7fbbcc0c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 --2- 172.21.15.33:0/2995411411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbcc06a210 0x7fbbcc147530 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 >> 172.21.15.33:0/2995411411 conn(0x7fbbcc0cc6b0 msgr2=0x7fbbcc153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 shutdown_connections 2023-10-06T02:45:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.385+0000 7fbbcbfff700 1 -- 172.21.15.33:0/2995411411 wait complete. 2023-10-06T02:45:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:45:42.406 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-10-06T02:45:40.564155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381960,"kb_used_data":3656,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592760,"statfs":{"total":773094113280,"available":770654986240,"internally_reserved":0,"allocated":3743744,"data_stored":2209232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004861"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.352797+0000","last_change":"2023-10-06T02:45:27.343811+0000","last_active":"2023-10-06T02:45:27.352797+0000","last_peered":"2023-10-06T02:45:27.352797+0000","last_clean":"2023-10-06T02:45:27.352797+0000","last_became_active":"2023-10-06T02:45:27.343175+0000","last_became_peered":"2023-10-06T02:45:27.343175+0000","last_unstale":"2023-10-06T02:45:27.352797+0000","last_undegraded":"2023-10-06T02:45:27.352797+0000","last_fullsized":"2023-10-06T02:45:27.352797+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334994+0000","last_change":"2023-10-06T02:45:22.801080+0000","last_active":"2023-10-06T02:45:27.334994+0000","last_peered":"2023-10-06T02:45:27.334994+0000","last_clean":"2023-10-06T02:45:27.334994+0000","last_became_active":"2023-10-06T02:45:22.800763+0000","last_became_peered":"2023-10-06T02:45:22.800763+0000","last_unstale":"2023-10-06T02:45:27.334994+0000","last_undegraded":"2023-10-06T02:45:27.334994+0000","last_fullsized":"2023-10-06T02:45:27.334994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333783+0000","last_change":"2023-10-06T02:45:23.305500+0000","last_active":"2023-10-06T02:45:27.333783+0000","last_peered":"2023-10-06T02:45:27.333783+0000","last_clean":"2023-10-06T02:45:27.333783+0000","last_became_active":"2023-10-06T02:45:23.305199+0000","last_became_peered":"2023-10-06T02:45:23.305199+0000","last_unstale":"2023-10-06T02:45:27.333783+0000","last_undegraded":"2023-10-06T02:45:27.333783+0000","last_fullsized":"2023-10-06T02:45:27.333783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351635+0000","last_change":"2023-10-06T02:45:27.345953+0000","last_active":"2023-10-06T02:45:27.351635+0000","last_peered":"2023-10-06T02:45:27.351635+0000","last_clean":"2023-10-06T02:45:27.351635+0000","last_became_active":"2023-10-06T02:45:27.345526+0000","last_became_peered":"2023-10-06T02:45:27.345526+0000","last_unstale":"2023-10-06T02:45:27.351635+0000","last_undegraded":"2023-10-06T02:45:27.351635+0000","last_fullsized":"2023-10-06T02:45:27.351635+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:37.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332907+0000","last_change":"2023-10-06T02:45:22.805277+0000","last_active":"2023-10-06T02:45:27.332907+0000","last_peered":"2023-10-06T02:45:27.332907+0000","last_clean":"2023-10-06T02:45:27.332907+0000","last_became_active":"2023-10-06T02:45:22.804802+0000","last_became_peered":"2023-10-06T02:45:22.804802+0000","last_unstale":"2023-10-06T02:45:27.332907+0000","last_undegraded":"2023-10-06T02:45:27.332907+0000","last_fullsized":"2023-10-06T02:45:27.332907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:37.831567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330139+0000","last_change":"2023-10-06T02:45:23.306937+0000","last_active":"2023-10-06T02:45:27.330139+0000","last_peered":"2023-10-06T02:45:27.330139+0000","last_clean":"2023-10-06T02:45:27.330139+0000","last_became_active":"2023-10-06T02:45:23.306641+0000","last_became_peered":"2023-10-06T02:45:23.306641+0000","last_unstale":"2023-10-06T02:45:27.330139+0000","last_undegraded":"2023-10-06T02:45:27.330139+0000","last_fullsized":"2023-10-06T02:45:27.330139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:23.162123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356992+0000","last_change":"2023-10-06T02:45:27.353931+0000","last_active":"2023-10-06T02:45:27.356992+0000","last_peered":"2023-10-06T02:45:27.356992+0000","last_clean":"2023-10-06T02:45:27.356992+0000","last_became_active":"2023-10-06T02:45:27.353488+0000","last_became_peered":"2023-10-06T02:45:27.353488+0000","last_unstale":"2023-10-06T02:45:27.356992+0000","last_undegraded":"2023-10-06T02:45:27.356992+0000","last_fullsized":"2023-10-06T02:45:27.356992+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341505+0000","last_change":"2023-10-06T02:45:22.803357+0000","last_active":"2023-10-06T02:45:27.341505+0000","last_peered":"2023-10-06T02:45:27.341505+0000","last_clean":"2023-10-06T02:45:27.341505+0000","last_became_active":"2023-10-06T02:45:22.800673+0000","last_became_peered":"2023-10-06T02:45:22.800673+0000","last_unstale":"2023-10-06T02:45:27.341505+0000","last_undegraded":"2023-10-06T02:45:27.341505+0000","last_fullsized":"2023-10-06T02:45:27.341505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336470+0000","last_change":"2023-10-06T02:45:23.310269+0000","last_active":"2023-10-06T02:45:27.336470+0000","last_peered":"2023-10-06T02:45:27.336470+0000","last_clean":"2023-10-06T02:45:27.336470+0000","last_became_active":"2023-10-06T02:45:23.309927+0000","last_became_peered":"2023-10-06T02:45:23.309927+0000","last_unstale":"2023-10-06T02:45:27.336470+0000","last_undegraded":"2023-10-06T02:45:27.336470+0000","last_fullsized":"2023-10-06T02:45:27.336470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351290+0000","last_change":"2023-10-06T02:45:27.348258+0000","last_active":"2023-10-06T02:45:27.351290+0000","last_peered":"2023-10-06T02:45:27.351290+0000","last_clean":"2023-10-06T02:45:27.351290+0000","last_became_active":"2023-10-06T02:45:27.347848+0000","last_became_peered":"2023-10-06T02:45:27.347848+0000","last_unstale":"2023-10-06T02:45:27.351290+0000","last_undegraded":"2023-10-06T02:45:27.351290+0000","last_fullsized":"2023-10-06T02:45:27.351290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340794+0000","last_change":"2023-10-06T02:45:22.800086+0000","last_active":"2023-10-06T02:45:27.340794+0000","last_peered":"2023-10-06T02:45:27.340794+0000","last_clean":"2023-10-06T02:45:27.340794+0000","last_became_active":"2023-10-06T02:45:22.799739+0000","last_became_peered":"2023-10-06T02:45:22.799739+0000","last_unstale":"2023-10-06T02:45:27.340794+0000","last_undegraded":"2023-10-06T02:45:27.340794+0000","last_fullsized":"2023-10-06T02:45:27.340794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332497+0000","last_change":"2023-10-06T02:45:23.312373+0000","last_active":"2023-10-06T02:45:27.332497+0000","last_peered":"2023-10-06T02:45:27.332497+0000","last_clean":"2023-10-06T02:45:27.332497+0000","last_became_active":"2023-10-06T02:45:23.311513+0000","last_became_peered":"2023-10-06T02:45:23.311513+0000","last_unstale":"2023-10-06T02:45:27.332497+0000","last_undegraded":"2023-10-06T02:45:27.332497+0000","last_fullsized":"2023-10-06T02:45:27.332497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:10.715925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360984+0000","last_change":"2023-10-06T02:45:27.347884+0000","last_active":"2023-10-06T02:45:27.360984+0000","last_peered":"2023-10-06T02:45:27.360984+0000","last_clean":"2023-10-06T02:45:27.360984+0000","last_became_active":"2023-10-06T02:45:27.347285+0000","last_became_peered":"2023-10-06T02:45:27.347285+0000","last_unstale":"2023-10-06T02:45:27.360984+0000","last_undegraded":"2023-10-06T02:45:27.360984+0000","last_fullsized":"2023-10-06T02:45:27.360984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:52.629371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335221+0000","last_change":"2023-10-06T02:45:22.804322+0000","last_active":"2023-10-06T02:45:27.335221+0000","last_peered":"2023-10-06T02:45:27.335221+0000","last_clean":"2023-10-06T02:45:27.335221+0000","last_became_active":"2023-10-06T02:45:22.802485+0000","last_became_peered":"2023-10-06T02:45:22.802485+0000","last_unstale":"2023-10-06T02:45:27.335221+0000","last_undegraded":"2023-10-06T02:45:27.335221+0000","last_fullsized":"2023-10-06T02:45:27.335221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320001+0000","last_change":"2023-10-06T02:45:23.311929+0000","last_active":"2023-10-06T02:45:27.320001+0000","last_peered":"2023-10-06T02:45:27.320001+0000","last_clean":"2023-10-06T02:45:27.320001+0000","last_became_active":"2023-10-06T02:45:23.311209+0000","last_became_peered":"2023-10-06T02:45:23.311209+0000","last_unstale":"2023-10-06T02:45:27.320001+0000","last_undegraded":"2023-10-06T02:45:27.320001+0000","last_fullsized":"2023-10-06T02:45:27.320001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:39.089521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354213+0000","last_change":"2023-10-06T02:45:27.347958+0000","last_active":"2023-10-06T02:45:27.354213+0000","last_peered":"2023-10-06T02:45:27.354213+0000","last_clean":"2023-10-06T02:45:27.354213+0000","last_became_active":"2023-10-06T02:45:27.347490+0000","last_became_peered":"2023-10-06T02:45:27.347490+0000","last_unstale":"2023-10-06T02:45:27.354213+0000","last_undegraded":"2023-10-06T02:45:27.354213+0000","last_fullsized":"2023-10-06T02:45:27.354213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:40.393467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336567+0000","last_change":"2023-10-06T02:45:22.799339+0000","last_active":"2023-10-06T02:45:27.336567+0000","last_peered":"2023-10-06T02:45:27.336567+0000","last_clean":"2023-10-06T02:45:27.336567+0000","last_became_active":"2023-10-06T02:45:22.798982+0000","last_became_peered":"2023-10-06T02:45:22.798982+0000","last_unstale":"2023-10-06T02:45:27.336567+0000","last_undegraded":"2023-10-06T02:45:27.336567+0000","last_fullsized":"2023-10-06T02:45:27.336567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:41.206862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330449+0000","last_change":"2023-10-06T02:45:23.312269+0000","last_active":"2023-10-06T02:45:27.330449+0000","last_peered":"2023-10-06T02:45:27.330449+0000","last_clean":"2023-10-06T02:45:27.330449+0000","last_became_active":"2023-10-06T02:45:23.311975+0000","last_became_peered":"2023-10-06T02:45:23.311975+0000","last_unstale":"2023-10-06T02:45:27.330449+0000","last_undegraded":"2023-10-06T02:45:27.330449+0000","last_fullsized":"2023-10-06T02:45:27.330449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:27.851854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354212+0000","last_change":"2023-10-06T02:45:27.347553+0000","last_active":"2023-10-06T02:45:27.354212+0000","last_peered":"2023-10-06T02:45:27.354212+0000","last_clean":"2023-10-06T02:45:27.354212+0000","last_became_active":"2023-10-06T02:45:27.347165+0000","last_became_peered":"2023-10-06T02:45:27.347165+0000","last_unstale":"2023-10-06T02:45:27.354212+0000","last_undegraded":"2023-10-06T02:45:27.354212+0000","last_fullsized":"2023-10-06T02:45:27.354212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:38.681094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:22.808457+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:22.808145+0000","last_became_peered":"2023-10-06T02:45:22.808145+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334405+0000","last_change":"2023-10-06T02:45:23.323552+0000","last_active":"2023-10-06T02:45:27.334405+0000","last_peered":"2023-10-06T02:45:27.334405+0000","last_clean":"2023-10-06T02:45:27.334405+0000","last_became_active":"2023-10-06T02:45:23.323241+0000","last_became_peered":"2023-10-06T02:45:23.323241+0000","last_unstale":"2023-10-06T02:45:27.334405+0000","last_undegraded":"2023-10-06T02:45:27.334405+0000","last_fullsized":"2023-10-06T02:45:27.334405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353329+0000","last_change":"2023-10-06T02:45:27.349343+0000","last_active":"2023-10-06T02:45:27.353329+0000","last_peered":"2023-10-06T02:45:27.353329+0000","last_clean":"2023-10-06T02:45:27.353329+0000","last_became_active":"2023-10-06T02:45:27.348859+0000","last_became_peered":"2023-10-06T02:45:27.348859+0000","last_unstale":"2023-10-06T02:45:27.353329+0000","last_undegraded":"2023-10-06T02:45:27.353329+0000","last_fullsized":"2023-10-06T02:45:27.353329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333023+0000","last_change":"2023-10-06T02:45:22.811866+0000","last_active":"2023-10-06T02:45:27.333023+0000","last_peered":"2023-10-06T02:45:27.333023+0000","last_clean":"2023-10-06T02:45:27.333023+0000","last_became_active":"2023-10-06T02:45:22.811334+0000","last_became_peered":"2023-10-06T02:45:22.811334+0000","last_unstale":"2023-10-06T02:45:27.333023+0000","last_undegraded":"2023-10-06T02:45:27.333023+0000","last_fullsized":"2023-10-06T02:45:27.333023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325429+0000","last_change":"2023-10-06T02:45:23.320977+0000","last_active":"2023-10-06T02:45:27.325429+0000","last_peered":"2023-10-06T02:45:27.325429+0000","last_clean":"2023-10-06T02:45:27.325429+0000","last_became_active":"2023-10-06T02:45:23.320677+0000","last_became_peered":"2023-10-06T02:45:23.320677+0000","last_unstale":"2023-10-06T02:45:27.325429+0000","last_undegraded":"2023-10-06T02:45:27.325429+0000","last_fullsized":"2023-10-06T02:45:27.325429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:18.520846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332892+0000","last_change":"2023-10-06T02:45:23.312712+0000","last_active":"2023-10-06T02:45:27.332892+0000","last_peered":"2023-10-06T02:45:27.332892+0000","last_clean":"2023-10-06T02:45:27.332892+0000","last_became_active":"2023-10-06T02:45:23.312164+0000","last_became_peered":"2023-10-06T02:45:23.312164+0000","last_unstale":"2023-10-06T02:45:27.332892+0000","last_undegraded":"2023-10-06T02:45:27.332892+0000","last_fullsized":"2023-10-06T02:45:27.332892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.186960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335089+0000","last_change":"2023-10-06T02:45:22.800781+0000","last_active":"2023-10-06T02:45:27.335089+0000","last_peered":"2023-10-06T02:45:27.335089+0000","last_clean":"2023-10-06T02:45:27.335089+0000","last_became_active":"2023-10-06T02:45:22.800484+0000","last_became_peered":"2023-10-06T02:45:22.800484+0000","last_unstale":"2023-10-06T02:45:27.335089+0000","last_undegraded":"2023-10-06T02:45:27.335089+0000","last_fullsized":"2023-10-06T02:45:27.335089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354176+0000","last_change":"2023-10-06T02:45:27.348699+0000","last_active":"2023-10-06T02:45:27.354176+0000","last_peered":"2023-10-06T02:45:27.354176+0000","last_clean":"2023-10-06T02:45:27.354176+0000","last_became_active":"2023-10-06T02:45:27.348111+0000","last_became_peered":"2023-10-06T02:45:27.348111+0000","last_unstale":"2023-10-06T02:45:27.354176+0000","last_undegraded":"2023-10-06T02:45:27.354176+0000","last_fullsized":"2023-10-06T02:45:27.354176+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:26.404420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330041+0000","last_change":"2023-10-06T02:45:23.323471+0000","last_active":"2023-10-06T02:45:27.330041+0000","last_peered":"2023-10-06T02:45:27.330041+0000","last_clean":"2023-10-06T02:45:27.330041+0000","last_became_active":"2023-10-06T02:45:23.323203+0000","last_became_peered":"2023-10-06T02:45:23.323203+0000","last_unstale":"2023-10-06T02:45:27.330041+0000","last_undegraded":"2023-10-06T02:45:27.330041+0000","last_fullsized":"2023-10-06T02:45:27.330041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:59.446720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341624+0000","last_change":"2023-10-06T02:45:22.804313+0000","last_active":"2023-10-06T02:45:27.341624+0000","last_peered":"2023-10-06T02:45:27.341624+0000","last_clean":"2023-10-06T02:45:27.341624+0000","last_became_active":"2023-10-06T02:45:22.804008+0000","last_became_peered":"2023-10-06T02:45:22.804008+0000","last_unstale":"2023-10-06T02:45:27.341624+0000","last_undegraded":"2023-10-06T02:45:27.341624+0000","last_fullsized":"2023-10-06T02:45:27.341624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356829+0000","last_change":"2023-10-06T02:45:27.355277+0000","last_active":"2023-10-06T02:45:27.356829+0000","last_peered":"2023-10-06T02:45:27.356829+0000","last_clean":"2023-10-06T02:45:27.356829+0000","last_became_active":"2023-10-06T02:45:27.354885+0000","last_became_peered":"2023-10-06T02:45:27.354885+0000","last_unstale":"2023-10-06T02:45:27.356829+0000","last_undegraded":"2023-10-06T02:45:27.356829+0000","last_fullsized":"2023-10-06T02:45:27.356829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.722369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333001+0000","last_change":"2023-10-06T02:45:22.810372+0000","last_active":"2023-10-06T02:45:27.333001+0000","last_peered":"2023-10-06T02:45:27.333001+0000","last_clean":"2023-10-06T02:45:27.333001+0000","last_became_active":"2023-10-06T02:45:22.810117+0000","last_became_peered":"2023-10-06T02:45:22.810117+0000","last_unstale":"2023-10-06T02:45:27.333001+0000","last_undegraded":"2023-10-06T02:45:27.333001+0000","last_fullsized":"2023-10-06T02:45:27.333001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:54.333503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335943+0000","last_change":"2023-10-06T02:45:23.319583+0000","last_active":"2023-10-06T02:45:27.335943+0000","last_peered":"2023-10-06T02:45:27.335943+0000","last_clean":"2023-10-06T02:45:27.335943+0000","last_became_active":"2023-10-06T02:45:23.319266+0000","last_became_peered":"2023-10-06T02:45:23.319266+0000","last_unstale":"2023-10-06T02:45:27.335943+0000","last_undegraded":"2023-10-06T02:45:27.335943+0000","last_fullsized":"2023-10-06T02:45:27.335943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353966+0000","last_change":"2023-10-06T02:45:27.351069+0000","last_active":"2023-10-06T02:45:27.353966+0000","last_peered":"2023-10-06T02:45:27.353966+0000","last_clean":"2023-10-06T02:45:27.353966+0000","last_became_active":"2023-10-06T02:45:27.350579+0000","last_became_peered":"2023-10-06T02:45:27.350579+0000","last_unstale":"2023-10-06T02:45:27.353966+0000","last_undegraded":"2023-10-06T02:45:27.353966+0000","last_fullsized":"2023-10-06T02:45:27.353966+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:47.948820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"29'9","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:33.113646+0000","last_change":"2023-10-06T02:45:22.804024+0000","last_active":"2023-10-06T02:45:33.113646+0000","last_peered":"2023-10-06T02:45:33.113646+0000","last_clean":"2023-10-06T02:45:33.113646+0000","last_became_active":"2023-10-06T02:45:22.803679+0000","last_became_peered":"2023-10-06T02:45:22.803679+0000","last_unstale":"2023-10-06T02:45:33.113646+0000","last_undegraded":"2023-10-06T02:45:33.113646+0000","last_fullsized":"2023-10-06T02:45:33.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.507771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332592+0000","last_change":"2023-10-06T02:45:23.312638+0000","last_active":"2023-10-06T02:45:27.332592+0000","last_peered":"2023-10-06T02:45:27.332592+0000","last_clean":"2023-10-06T02:45:27.332592+0000","last_became_active":"2023-10-06T02:45:23.312006+0000","last_became_peered":"2023-10-06T02:45:23.312006+0000","last_unstale":"2023-10-06T02:45:27.332592+0000","last_undegraded":"2023-10-06T02:45:27.332592+0000","last_fullsized":"2023-10-06T02:45:27.332592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.358568+0000","last_change":"2023-10-06T02:45:27.345474+0000","last_active":"2023-10-06T02:45:27.358568+0000","last_peered":"2023-10-06T02:45:27.358568+0000","last_clean":"2023-10-06T02:45:27.358568+0000","last_became_active":"2023-10-06T02:45:27.345014+0000","last_became_peered":"2023-10-06T02:45:27.345014+0000","last_unstale":"2023-10-06T02:45:27.358568+0000","last_undegraded":"2023-10-06T02:45:27.358568+0000","last_fullsized":"2023-10-06T02:45:27.358568+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334989+0000","last_change":"2023-10-06T02:45:23.319302+0000","last_active":"2023-10-06T02:45:27.334989+0000","last_peered":"2023-10-06T02:45:27.334989+0000","last_clean":"2023-10-06T02:45:27.334989+0000","last_became_active":"2023-10-06T02:45:23.318951+0000","last_became_peered":"2023-10-06T02:45:23.318951+0000","last_unstale":"2023-10-06T02:45:27.334989+0000","last_undegraded":"2023-10-06T02:45:27.334989+0000","last_fullsized":"2023-10-06T02:45:27.334989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:34.946622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325697+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:45:27.325697+0000","last_peered":"2023-10-06T02:45:27.325697+0000","last_clean":"2023-10-06T02:45:27.325697+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:45:27.325697+0000","last_undegraded":"2023-10-06T02:45:27.325697+0000","last_fullsized":"2023-10-06T02:45:27.325697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:28.296097+0000","last_change":"2023-10-06T02:45:22.799764+0000","last_active":"2023-10-06T02:45:28.296097+0000","last_peered":"2023-10-06T02:45:28.296097+0000","last_clean":"2023-10-06T02:45:28.296097+0000","last_became_active":"2023-10-06T02:45:22.798917+0000","last_became_peered":"2023-10-06T02:45:22.798917+0000","last_unstale":"2023-10-06T02:45:28.296097+0000","last_undegraded":"2023-10-06T02:45:28.296097+0000","last_fullsized":"2023-10-06T02:45:28.296097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353993+0000","last_change":"2023-10-06T02:45:27.350164+0000","last_active":"2023-10-06T02:45:27.353993+0000","last_peered":"2023-10-06T02:45:27.353993+0000","last_clean":"2023-10-06T02:45:27.353993+0000","last_became_active":"2023-10-06T02:45:27.349731+0000","last_became_peered":"2023-10-06T02:45:27.349731+0000","last_unstale":"2023-10-06T02:45:27.353993+0000","last_undegraded":"2023-10-06T02:45:27.353993+0000","last_fullsized":"2023-10-06T02:45:27.353993+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:07.179714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332977+0000","last_change":"2023-10-06T02:45:23.324382+0000","last_active":"2023-10-06T02:45:27.332977+0000","last_peered":"2023-10-06T02:45:27.332977+0000","last_clean":"2023-10-06T02:45:27.332977+0000","last_became_active":"2023-10-06T02:45:23.324017+0000","last_became_peered":"2023-10-06T02:45:23.324017+0000","last_unstale":"2023-10-06T02:45:27.332977+0000","last_undegraded":"2023-10-06T02:45:27.332977+0000","last_fullsized":"2023-10-06T02:45:27.332977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:01.347856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340460+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:45:27.340460+0000","last_peered":"2023-10-06T02:45:27.340460+0000","last_clean":"2023-10-06T02:45:27.340460+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:45:27.340460+0000","last_undegraded":"2023-10-06T02:45:27.340460+0000","last_fullsized":"2023-10-06T02:45:27.340460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333588+0000","last_change":"2023-10-06T02:45:22.807333+0000","last_active":"2023-10-06T02:45:27.333588+0000","last_peered":"2023-10-06T02:45:27.333588+0000","last_clean":"2023-10-06T02:45:27.333588+0000","last_became_active":"2023-10-06T02:45:22.805988+0000","last_became_peered":"2023-10-06T02:45:22.805988+0000","last_unstale":"2023-10-06T02:45:27.333588+0000","last_undegraded":"2023-10-06T02:45:27.333588+0000","last_fullsized":"2023-10-06T02:45:27.333588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:41.870262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353464+0000","last_change":"2023-10-06T02:45:27.347859+0000","last_active":"2023-10-06T02:45:27.353464+0000","last_peered":"2023-10-06T02:45:27.353464+0000","last_clean":"2023-10-06T02:45:27.353464+0000","last_became_active":"2023-10-06T02:45:27.347418+0000","last_became_peered":"2023-10-06T02:45:27.347418+0000","last_unstale":"2023-10-06T02:45:27.353464+0000","last_undegraded":"2023-10-06T02:45:27.353464+0000","last_fullsized":"2023-10-06T02:45:27.353464+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:17.541971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334956+0000","last_change":"2023-10-06T02:45:23.320296+0000","last_active":"2023-10-06T02:45:27.334956+0000","last_peered":"2023-10-06T02:45:27.334956+0000","last_clean":"2023-10-06T02:45:27.334956+0000","last_became_active":"2023-10-06T02:45:23.319983+0000","last_became_peered":"2023-10-06T02:45:23.319983+0000","last_unstale":"2023-10-06T02:45:27.334956+0000","last_undegraded":"2023-10-06T02:45:27.334956+0000","last_fullsized":"2023-10-06T02:45:27.334956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333252+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:45:27.333252+0000","last_peered":"2023-10-06T02:45:27.333252+0000","last_clean":"2023-10-06T02:45:27.333252+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:45:27.333252+0000","last_undegraded":"2023-10-06T02:45:27.333252+0000","last_fullsized":"2023-10-06T02:45:27.333252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336685+0000","last_change":"2023-10-06T02:45:22.804019+0000","last_active":"2023-10-06T02:45:27.336685+0000","last_peered":"2023-10-06T02:45:27.336685+0000","last_clean":"2023-10-06T02:45:27.336685+0000","last_became_active":"2023-10-06T02:45:22.803524+0000","last_became_peered":"2023-10-06T02:45:22.803524+0000","last_unstale":"2023-10-06T02:45:27.336685+0000","last_undegraded":"2023-10-06T02:45:27.336685+0000","last_fullsized":"2023-10-06T02:45:27.336685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353592+0000","last_change":"2023-10-06T02:45:27.344027+0000","last_active":"2023-10-06T02:45:27.353592+0000","last_peered":"2023-10-06T02:45:27.353592+0000","last_clean":"2023-10-06T02:45:27.353592+0000","last_became_active":"2023-10-06T02:45:27.343450+0000","last_became_peered":"2023-10-06T02:45:27.343450+0000","last_unstale":"2023-10-06T02:45:27.353592+0000","last_undegraded":"2023-10-06T02:45:27.353592+0000","last_fullsized":"2023-10-06T02:45:27.353592+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:42.951337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336038+0000","last_change":"2023-10-06T02:45:23.319757+0000","last_active":"2023-10-06T02:45:27.336038+0000","last_peered":"2023-10-06T02:45:27.336038+0000","last_clean":"2023-10-06T02:45:27.336038+0000","last_became_active":"2023-10-06T02:45:23.319356+0000","last_became_peered":"2023-10-06T02:45:23.319356+0000","last_unstale":"2023-10-06T02:45:27.336038+0000","last_undegraded":"2023-10-06T02:45:27.336038+0000","last_fullsized":"2023-10-06T02:45:27.336038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337310+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:45:27.337310+0000","last_peered":"2023-10-06T02:45:27.337310+0000","last_clean":"2023-10-06T02:45:27.337310+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:45:27.337310+0000","last_undegraded":"2023-10-06T02:45:27.337310+0000","last_fullsized":"2023-10-06T02:45:27.337310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341535+0000","last_change":"2023-10-06T02:45:22.808675+0000","last_active":"2023-10-06T02:45:27.341535+0000","last_peered":"2023-10-06T02:45:27.341535+0000","last_clean":"2023-10-06T02:45:27.341535+0000","last_became_active":"2023-10-06T02:45:22.807693+0000","last_became_peered":"2023-10-06T02:45:22.807693+0000","last_unstale":"2023-10-06T02:45:27.341535+0000","last_undegraded":"2023-10-06T02:45:27.341535+0000","last_fullsized":"2023-10-06T02:45:27.341535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356445+0000","last_change":"2023-10-06T02:45:27.352852+0000","last_active":"2023-10-06T02:45:27.356445+0000","last_peered":"2023-10-06T02:45:27.356445+0000","last_clean":"2023-10-06T02:45:27.356445+0000","last_became_active":"2023-10-06T02:45:27.352303+0000","last_became_peered":"2023-10-06T02:45:27.352303+0000","last_unstale":"2023-10-06T02:45:27.356445+0000","last_undegraded":"2023-10-06T02:45:27.356445+0000","last_fullsized":"2023-10-06T02:45:27.356445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:21.854256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336597+0000","last_change":"2023-10-06T02:45:23.317570+0000","last_active":"2023-10-06T02:45:27.336597+0000","last_peered":"2023-10-06T02:45:27.336597+0000","last_clean":"2023-10-06T02:45:27.336597+0000","last_became_active":"2023-10-06T02:45:23.316419+0000","last_became_peered":"2023-10-06T02:45:23.316419+0000","last_unstale":"2023-10-06T02:45:27.336597+0000","last_undegraded":"2023-10-06T02:45:27.336597+0000","last_fullsized":"2023-10-06T02:45:27.336597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332685+0000","last_change":"2023-10-06T02:45:22.799727+0000","last_active":"2023-10-06T02:45:27.332685+0000","last_peered":"2023-10-06T02:45:27.332685+0000","last_clean":"2023-10-06T02:45:27.332685+0000","last_became_active":"2023-10-06T02:45:22.799276+0000","last_became_peered":"2023-10-06T02:45:22.799276+0000","last_unstale":"2023-10-06T02:45:27.332685+0000","last_undegraded":"2023-10-06T02:45:27.332685+0000","last_fullsized":"2023-10-06T02:45:27.332685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340308+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:45:27.340308+0000","last_peered":"2023-10-06T02:45:27.340308+0000","last_clean":"2023-10-06T02:45:27.340308+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:45:27.340308+0000","last_undegraded":"2023-10-06T02:45:27.340308+0000","last_fullsized":"2023-10-06T02:45:27.340308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353669+0000","last_change":"2023-10-06T02:45:27.350898+0000","last_active":"2023-10-06T02:45:27.353669+0000","last_peered":"2023-10-06T02:45:27.353669+0000","last_clean":"2023-10-06T02:45:27.353669+0000","last_became_active":"2023-10-06T02:45:27.350489+0000","last_became_peered":"2023-10-06T02:45:27.350489+0000","last_unstale":"2023-10-06T02:45:27.353669+0000","last_undegraded":"2023-10-06T02:45:27.353669+0000","last_fullsized":"2023-10-06T02:45:27.353669+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340516+0000","last_change":"2023-10-06T02:45:23.320581+0000","last_active":"2023-10-06T02:45:27.340516+0000","last_peered":"2023-10-06T02:45:27.340516+0000","last_clean":"2023-10-06T02:45:27.340516+0000","last_became_active":"2023-10-06T02:45:23.320253+0000","last_became_peered":"2023-10-06T02:45:23.320253+0000","last_unstale":"2023-10-06T02:45:27.340516+0000","last_undegraded":"2023-10-06T02:45:27.340516+0000","last_fullsized":"2023-10-06T02:45:27.340516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:15.466869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334594+0000","last_change":"2023-10-06T02:45:22.803922+0000","last_active":"2023-10-06T02:45:27.334594+0000","last_peered":"2023-10-06T02:45:27.334594+0000","last_clean":"2023-10-06T02:45:27.334594+0000","last_became_active":"2023-10-06T02:45:22.803487+0000","last_became_peered":"2023-10-06T02:45:22.803487+0000","last_unstale":"2023-10-06T02:45:27.334594+0000","last_undegraded":"2023-10-06T02:45:27.334594+0000","last_fullsized":"2023-10-06T02:45:27.334594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319371+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:45:27.319371+0000","last_peered":"2023-10-06T02:45:27.319371+0000","last_clean":"2023-10-06T02:45:27.319371+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:45:27.319371+0000","last_undegraded":"2023-10-06T02:45:27.319371+0000","last_fullsized":"2023-10-06T02:45:27.319371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350330+0000","last_change":"2023-10-06T02:45:27.341548+0000","last_active":"2023-10-06T02:45:27.350330+0000","last_peered":"2023-10-06T02:45:27.350330+0000","last_clean":"2023-10-06T02:45:27.350330+0000","last_became_active":"2023-10-06T02:45:27.341116+0000","last_became_peered":"2023-10-06T02:45:27.341116+0000","last_unstale":"2023-10-06T02:45:27.350330+0000","last_undegraded":"2023-10-06T02:45:27.350330+0000","last_fullsized":"2023-10-06T02:45:27.350330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:12.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334501+0000","last_change":"2023-10-06T02:45:23.313236+0000","last_active":"2023-10-06T02:45:27.334501+0000","last_peered":"2023-10-06T02:45:27.334501+0000","last_clean":"2023-10-06T02:45:27.334501+0000","last_became_active":"2023-10-06T02:45:23.312943+0000","last_became_peered":"2023-10-06T02:45:23.312943+0000","last_unstale":"2023-10-06T02:45:27.334501+0000","last_undegraded":"2023-10-06T02:45:27.334501+0000","last_fullsized":"2023-10-06T02:45:27.334501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333980+0000","last_change":"2023-10-06T02:45:22.801183+0000","last_active":"2023-10-06T02:45:27.333980+0000","last_peered":"2023-10-06T02:45:27.333980+0000","last_clean":"2023-10-06T02:45:27.333980+0000","last_became_active":"2023-10-06T02:45:22.800872+0000","last_became_peered":"2023-10-06T02:45:22.800872+0000","last_unstale":"2023-10-06T02:45:27.333980+0000","last_undegraded":"2023-10-06T02:45:27.333980+0000","last_fullsized":"2023-10-06T02:45:27.333980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331616+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:45:27.331616+0000","last_peered":"2023-10-06T02:45:27.331616+0000","last_clean":"2023-10-06T02:45:27.331616+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:45:27.331616+0000","last_undegraded":"2023-10-06T02:45:27.331616+0000","last_fullsized":"2023-10-06T02:45:27.331616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357435+0000","last_change":"2023-10-06T02:45:27.354883+0000","last_active":"2023-10-06T02:45:27.357435+0000","last_peered":"2023-10-06T02:45:27.357435+0000","last_clean":"2023-10-06T02:45:27.357435+0000","last_became_active":"2023-10-06T02:45:27.353904+0000","last_became_peered":"2023-10-06T02:45:27.353904+0000","last_unstale":"2023-10-06T02:45:27.357435+0000","last_undegraded":"2023-10-06T02:45:27.357435+0000","last_fullsized":"2023-10-06T02:45:27.357435+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:54.845793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325620+0000","last_change":"2023-10-06T02:45:23.311475+0000","last_active":"2023-10-06T02:45:27.325620+0000","last_peered":"2023-10-06T02:45:27.325620+0000","last_clean":"2023-10-06T02:45:27.325620+0000","last_became_active":"2023-10-06T02:45:23.311183+0000","last_became_peered":"2023-10-06T02:45:23.311183+0000","last_unstale":"2023-10-06T02:45:27.325620+0000","last_undegraded":"2023-10-06T02:45:27.325620+0000","last_fullsized":"2023-10-06T02:45:27.325620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:38.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331867+0000","last_change":"2023-10-06T02:45:22.800538+0000","last_active":"2023-10-06T02:45:27.331867+0000","last_peered":"2023-10-06T02:45:27.331867+0000","last_clean":"2023-10-06T02:45:27.331867+0000","last_became_active":"2023-10-06T02:45:22.800251+0000","last_became_peered":"2023-10-06T02:45:22.800251+0000","last_unstale":"2023-10-06T02:45:27.331867+0000","last_undegraded":"2023-10-06T02:45:27.331867+0000","last_fullsized":"2023-10-06T02:45:27.331867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:35.553098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340818+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:45:27.340818+0000","last_peered":"2023-10-06T02:45:27.340818+0000","last_clean":"2023-10-06T02:45:27.340818+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:45:27.340818+0000","last_undegraded":"2023-10-06T02:45:27.340818+0000","last_fullsized":"2023-10-06T02:45:27.340818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331708+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:45:27.331708+0000","last_peered":"2023-10-06T02:45:27.331708+0000","last_clean":"2023-10-06T02:45:27.331708+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:45:27.331708+0000","last_undegraded":"2023-10-06T02:45:27.331708+0000","last_fullsized":"2023-10-06T02:45:27.331708+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351388+0000","last_change":"2023-10-06T02:45:27.349035+0000","last_active":"2023-10-06T02:45:27.351388+0000","last_peered":"2023-10-06T02:45:27.351388+0000","last_clean":"2023-10-06T02:45:27.351388+0000","last_became_active":"2023-10-06T02:45:27.348608+0000","last_became_peered":"2023-10-06T02:45:27.348608+0000","last_unstale":"2023-10-06T02:45:27.351388+0000","last_undegraded":"2023-10-06T02:45:27.351388+0000","last_fullsized":"2023-10-06T02:45:27.351388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:18.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340259+0000","last_change":"2023-10-06T02:45:23.310743+0000","last_active":"2023-10-06T02:45:27.340259+0000","last_peered":"2023-10-06T02:45:27.340259+0000","last_clean":"2023-10-06T02:45:27.340259+0000","last_became_active":"2023-10-06T02:45:23.310401+0000","last_became_peered":"2023-10-06T02:45:23.310401+0000","last_unstale":"2023-10-06T02:45:27.340259+0000","last_undegraded":"2023-10-06T02:45:27.340259+0000","last_fullsized":"2023-10-06T02:45:27.340259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333057+0000","last_change":"2023-10-06T02:45:22.807407+0000","last_active":"2023-10-06T02:45:27.333057+0000","last_peered":"2023-10-06T02:45:27.333057+0000","last_clean":"2023-10-06T02:45:27.333057+0000","last_became_active":"2023-10-06T02:45:22.804974+0000","last_became_peered":"2023-10-06T02:45:22.804974+0000","last_unstale":"2023-10-06T02:45:27.333057+0000","last_undegraded":"2023-10-06T02:45:27.333057+0000","last_fullsized":"2023-10-06T02:45:27.333057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:15.722413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351625+0000","last_change":"2023-10-06T02:45:27.346225+0000","last_active":"2023-10-06T02:45:27.351625+0000","last_peered":"2023-10-06T02:45:27.351625+0000","last_clean":"2023-10-06T02:45:27.351625+0000","last_became_active":"2023-10-06T02:45:27.345701+0000","last_became_peered":"2023-10-06T02:45:27.345701+0000","last_unstale":"2023-10-06T02:45:27.351625+0000","last_undegraded":"2023-10-06T02:45:27.351625+0000","last_fullsized":"2023-10-06T02:45:27.351625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:56.453178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334551+0000","last_change":"2023-10-06T02:45:23.310126+0000","last_active":"2023-10-06T02:45:27.334551+0000","last_peered":"2023-10-06T02:45:27.334551+0000","last_clean":"2023-10-06T02:45:27.334551+0000","last_became_active":"2023-10-06T02:45:23.309805+0000","last_became_peered":"2023-10-06T02:45:23.309805+0000","last_unstale":"2023-10-06T02:45:27.334551+0000","last_undegraded":"2023-10-06T02:45:27.334551+0000","last_fullsized":"2023-10-06T02:45:27.334551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336853+0000","last_change":"2023-10-06T02:45:22.804564+0000","last_active":"2023-10-06T02:45:27.336853+0000","last_peered":"2023-10-06T02:45:27.336853+0000","last_clean":"2023-10-06T02:45:27.336853+0000","last_became_active":"2023-10-06T02:45:22.804286+0000","last_became_peered":"2023-10-06T02:45:22.804286+0000","last_unstale":"2023-10-06T02:45:27.336853+0000","last_undegraded":"2023-10-06T02:45:27.336853+0000","last_fullsized":"2023-10-06T02:45:27.336853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357099+0000","last_change":"2023-10-06T02:45:27.354769+0000","last_active":"2023-10-06T02:45:27.357099+0000","last_peered":"2023-10-06T02:45:27.357099+0000","last_clean":"2023-10-06T02:45:27.357099+0000","last_became_active":"2023-10-06T02:45:27.352768+0000","last_became_peered":"2023-10-06T02:45:27.352768+0000","last_unstale":"2023-10-06T02:45:27.357099+0000","last_undegraded":"2023-10-06T02:45:27.357099+0000","last_fullsized":"2023-10-06T02:45:27.357099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336239+0000","last_change":"2023-10-06T02:45:23.310059+0000","last_active":"2023-10-06T02:45:27.336239+0000","last_peered":"2023-10-06T02:45:27.336239+0000","last_clean":"2023-10-06T02:45:27.336239+0000","last_became_active":"2023-10-06T02:45:23.309667+0000","last_became_peered":"2023-10-06T02:45:23.309667+0000","last_unstale":"2023-10-06T02:45:27.336239+0000","last_undegraded":"2023-10-06T02:45:27.336239+0000","last_fullsized":"2023-10-06T02:45:27.336239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:28.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.326136+0000","last_change":"2023-10-06T02:45:22.795986+0000","last_active":"2023-10-06T02:45:27.326136+0000","last_peered":"2023-10-06T02:45:27.326136+0000","last_clean":"2023-10-06T02:45:27.326136+0000","last_became_active":"2023-10-06T02:45:22.795649+0000","last_became_peered":"2023-10-06T02:45:22.795649+0000","last_unstale":"2023-10-06T02:45:27.326136+0000","last_undegraded":"2023-10-06T02:45:27.326136+0000","last_fullsized":"2023-10-06T02:45:27.326136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:52.473292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360933+0000","last_change":"2023-10-06T02:45:27.347788+0000","last_active":"2023-10-06T02:45:27.360933+0000","last_peered":"2023-10-06T02:45:27.360933+0000","last_clean":"2023-10-06T02:45:27.360933+0000","last_became_active":"2023-10-06T02:45:27.347192+0000","last_became_peered":"2023-10-06T02:45:27.347192+0000","last_unstale":"2023-10-06T02:45:27.360933+0000","last_undegraded":"2023-10-06T02:45:27.360933+0000","last_fullsized":"2023-10-06T02:45:27.360933+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336639+0000","last_change":"2023-10-06T02:45:23.307192+0000","last_active":"2023-10-06T02:45:27.336639+0000","last_peered":"2023-10-06T02:45:27.336639+0000","last_clean":"2023-10-06T02:45:27.336639+0000","last_became_active":"2023-10-06T02:45:23.306642+0000","last_became_peered":"2023-10-06T02:45:23.306642+0000","last_unstale":"2023-10-06T02:45:27.336639+0000","last_undegraded":"2023-10-06T02:45:27.336639+0000","last_fullsized":"2023-10-06T02:45:27.336639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:14.468949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333926+0000","last_change":"2023-10-06T02:45:22.811906+0000","last_active":"2023-10-06T02:45:27.333926+0000","last_peered":"2023-10-06T02:45:27.333926+0000","last_clean":"2023-10-06T02:45:27.333926+0000","last_became_active":"2023-10-06T02:45:22.811602+0000","last_became_peered":"2023-10-06T02:45:22.811602+0000","last_unstale":"2023-10-06T02:45:27.333926+0000","last_undegraded":"2023-10-06T02:45:27.333926+0000","last_fullsized":"2023-10-06T02:45:27.333926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.090764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.349516+0000","last_change":"2023-10-06T02:45:27.344016+0000","last_active":"2023-10-06T02:45:27.349516+0000","last_peered":"2023-10-06T02:45:27.349516+0000","last_clean":"2023-10-06T02:45:27.349516+0000","last_became_active":"2023-10-06T02:45:27.342950+0000","last_became_peered":"2023-10-06T02:45:27.342950+0000","last_unstale":"2023-10-06T02:45:27.349516+0000","last_undegraded":"2023-10-06T02:45:27.349516+0000","last_fullsized":"2023-10-06T02:45:27.349516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:25.726296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334869+0000","last_change":"2023-10-06T02:45:23.321013+0000","last_active":"2023-10-06T02:45:27.334869+0000","last_peered":"2023-10-06T02:45:27.334869+0000","last_clean":"2023-10-06T02:45:27.334869+0000","last_became_active":"2023-10-06T02:45:23.320682+0000","last_became_peered":"2023-10-06T02:45:23.320682+0000","last_unstale":"2023-10-06T02:45:27.334869+0000","last_undegraded":"2023-10-06T02:45:27.334869+0000","last_fullsized":"2023-10-06T02:45:27.334869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341800+0000","last_change":"2023-10-06T02:45:22.806416+0000","last_active":"2023-10-06T02:45:27.341800+0000","last_peered":"2023-10-06T02:45:27.341800+0000","last_clean":"2023-10-06T02:45:27.341800+0000","last_became_active":"2023-10-06T02:45:22.806087+0000","last_became_peered":"2023-10-06T02:45:22.806087+0000","last_unstale":"2023-10-06T02:45:27.341800+0000","last_undegraded":"2023-10-06T02:45:27.341800+0000","last_fullsized":"2023-10-06T02:45:27.341800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357937+0000","last_change":"2023-10-06T02:45:27.350897+0000","last_active":"2023-10-06T02:45:27.357937+0000","last_peered":"2023-10-06T02:45:27.357937+0000","last_clean":"2023-10-06T02:45:27.357937+0000","last_became_active":"2023-10-06T02:45:27.350497+0000","last_became_peered":"2023-10-06T02:45:27.350497+0000","last_unstale":"2023-10-06T02:45:27.357937+0000","last_undegraded":"2023-10-06T02:45:27.357937+0000","last_fullsized":"2023-10-06T02:45:27.357937+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:38.481448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325860+0000","last_change":"2023-10-06T02:45:23.313267+0000","last_active":"2023-10-06T02:45:27.325860+0000","last_peered":"2023-10-06T02:45:27.325860+0000","last_clean":"2023-10-06T02:45:27.325860+0000","last_became_active":"2023-10-06T02:45:23.312886+0000","last_became_peered":"2023-10-06T02:45:23.312886+0000","last_unstale":"2023-10-06T02:45:27.325860+0000","last_undegraded":"2023-10-06T02:45:27.325860+0000","last_fullsized":"2023-10-06T02:45:27.325860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:30.923831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333106+0000","last_change":"2023-10-06T02:45:22.809724+0000","last_active":"2023-10-06T02:45:27.333106+0000","last_peered":"2023-10-06T02:45:27.333106+0000","last_clean":"2023-10-06T02:45:27.333106+0000","last_became_active":"2023-10-06T02:45:22.809451+0000","last_became_peered":"2023-10-06T02:45:22.809451+0000","last_unstale":"2023-10-06T02:45:27.333106+0000","last_undegraded":"2023-10-06T02:45:27.333106+0000","last_fullsized":"2023-10-06T02:45:27.333106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:10.054721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.348579+0000","last_change":"2023-10-06T02:45:27.343402+0000","last_active":"2023-10-06T02:45:27.348579+0000","last_peered":"2023-10-06T02:45:27.348579+0000","last_clean":"2023-10-06T02:45:27.348579+0000","last_became_active":"2023-10-06T02:45:27.342892+0000","last_became_peered":"2023-10-06T02:45:27.342892+0000","last_unstale":"2023-10-06T02:45:27.348579+0000","last_undegraded":"2023-10-06T02:45:27.348579+0000","last_fullsized":"2023-10-06T02:45:27.348579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:05.116878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334800+0000","last_change":"2023-10-06T02:45:23.313977+0000","last_active":"2023-10-06T02:45:27.334800+0000","last_peered":"2023-10-06T02:45:27.334800+0000","last_clean":"2023-10-06T02:45:27.334800+0000","last_became_active":"2023-10-06T02:45:23.313656+0000","last_became_peered":"2023-10-06T02:45:23.313656+0000","last_unstale":"2023-10-06T02:45:27.334800+0000","last_undegraded":"2023-10-06T02:45:27.334800+0000","last_fullsized":"2023-10-06T02:45:27.334800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:52.328524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334307+0000","last_change":"2023-10-06T02:45:22.809096+0000","last_active":"2023-10-06T02:45:27.334307+0000","last_peered":"2023-10-06T02:45:27.334307+0000","last_clean":"2023-10-06T02:45:27.334307+0000","last_became_active":"2023-10-06T02:45:22.807367+0000","last_became_peered":"2023-10-06T02:45:22.807367+0000","last_unstale":"2023-10-06T02:45:27.334307+0000","last_undegraded":"2023-10-06T02:45:27.334307+0000","last_fullsized":"2023-10-06T02:45:27.334307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357950+0000","last_change":"2023-10-06T02:45:27.349954+0000","last_active":"2023-10-06T02:45:27.357950+0000","last_peered":"2023-10-06T02:45:27.357950+0000","last_clean":"2023-10-06T02:45:27.357950+0000","last_became_active":"2023-10-06T02:45:27.344601+0000","last_became_peered":"2023-10-06T02:45:27.344601+0000","last_unstale":"2023-10-06T02:45:27.357950+0000","last_undegraded":"2023-10-06T02:45:27.357950+0000","last_fullsized":"2023-10-06T02:45:27.357950+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:55.863350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340458+0000","last_change":"2023-10-06T02:45:23.325407+0000","last_active":"2023-10-06T02:45:27.340458+0000","last_peered":"2023-10-06T02:45:27.340458+0000","last_clean":"2023-10-06T02:45:27.340458+0000","last_became_active":"2023-10-06T02:45:23.325027+0000","last_became_peered":"2023-10-06T02:45:23.325027+0000","last_unstale":"2023-10-06T02:45:27.340458+0000","last_undegraded":"2023-10-06T02:45:27.340458+0000","last_fullsized":"2023-10-06T02:45:27.340458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334293+0000","last_change":"2023-10-06T02:45:22.807221+0000","last_active":"2023-10-06T02:45:27.334293+0000","last_peered":"2023-10-06T02:45:27.334293+0000","last_clean":"2023-10-06T02:45:27.334293+0000","last_became_active":"2023-10-06T02:45:22.806899+0000","last_became_peered":"2023-10-06T02:45:22.806899+0000","last_unstale":"2023-10-06T02:45:27.334293+0000","last_undegraded":"2023-10-06T02:45:27.334293+0000","last_fullsized":"2023-10-06T02:45:27.334293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356009+0000","last_change":"2023-10-06T02:45:27.352601+0000","last_active":"2023-10-06T02:45:27.356009+0000","last_peered":"2023-10-06T02:45:27.356009+0000","last_clean":"2023-10-06T02:45:27.356009+0000","last_became_active":"2023-10-06T02:45:27.352177+0000","last_became_peered":"2023-10-06T02:45:27.352177+0000","last_unstale":"2023-10-06T02:45:27.356009+0000","last_undegraded":"2023-10-06T02:45:27.356009+0000","last_fullsized":"2023-10-06T02:45:27.356009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:57.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357070+0000","last_change":"2023-10-06T02:45:27.354742+0000","last_active":"2023-10-06T02:45:27.357070+0000","last_peered":"2023-10-06T02:45:27.357070+0000","last_clean":"2023-10-06T02:45:27.357070+0000","last_became_active":"2023-10-06T02:45:27.354154+0000","last_became_peered":"2023-10-06T02:45:27.354154+0000","last_unstale":"2023-10-06T02:45:27.357070+0000","last_undegraded":"2023-10-06T02:45:27.357070+0000","last_fullsized":"2023-10-06T02:45:27.357070+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:03.091381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.327140+0000","last_change":"2023-10-06T02:45:22.802084+0000","last_active":"2023-10-06T02:45:27.327140+0000","last_peered":"2023-10-06T02:45:27.327140+0000","last_clean":"2023-10-06T02:45:27.327140+0000","last_became_active":"2023-10-06T02:45:22.801685+0000","last_became_peered":"2023-10-06T02:45:22.801685+0000","last_unstale":"2023-10-06T02:45:27.327140+0000","last_undegraded":"2023-10-06T02:45:27.327140+0000","last_fullsized":"2023-10-06T02:45:27.327140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:31.821785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333425+0000","last_change":"2023-10-06T02:45:23.323734+0000","last_active":"2023-10-06T02:45:27.333425+0000","last_peered":"2023-10-06T02:45:27.333425+0000","last_clean":"2023-10-06T02:45:27.333425+0000","last_became_active":"2023-10-06T02:45:23.323390+0000","last_became_peered":"2023-10-06T02:45:23.323390+0000","last_unstale":"2023-10-06T02:45:27.333425+0000","last_undegraded":"2023-10-06T02:45:27.333425+0000","last_fullsized":"2023-10-06T02:45:27.333425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:09:04.032805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350027+0000","last_change":"2023-10-06T02:45:27.346307+0000","last_active":"2023-10-06T02:45:27.350027+0000","last_peered":"2023-10-06T02:45:27.350027+0000","last_clean":"2023-10-06T02:45:27.350027+0000","last_became_active":"2023-10-06T02:45:27.345807+0000","last_became_peered":"2023-10-06T02:45:27.345807+0000","last_unstale":"2023-10-06T02:45:27.350027+0000","last_undegraded":"2023-10-06T02:45:27.350027+0000","last_fullsized":"2023-10-06T02:45:27.350027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.633950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332194+0000","last_change":"2023-10-06T02:45:22.809079+0000","last_active":"2023-10-06T02:45:27.332194+0000","last_peered":"2023-10-06T02:45:27.332194+0000","last_clean":"2023-10-06T02:45:27.332194+0000","last_became_active":"2023-10-06T02:45:22.807326+0000","last_became_peered":"2023-10-06T02:45:22.807326+0000","last_unstale":"2023-10-06T02:45:27.332194+0000","last_undegraded":"2023-10-06T02:45:27.332194+0000","last_fullsized":"2023-10-06T02:45:27.332194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:32.124040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334582+0000","last_change":"2023-10-06T02:45:23.313113+0000","last_active":"2023-10-06T02:45:27.334582+0000","last_peered":"2023-10-06T02:45:27.334582+0000","last_clean":"2023-10-06T02:45:27.334582+0000","last_became_active":"2023-10-06T02:45:23.312797+0000","last_became_peered":"2023-10-06T02:45:23.312797+0000","last_unstale":"2023-10-06T02:45:27.334582+0000","last_undegraded":"2023-10-06T02:45:27.334582+0000","last_fullsized":"2023-10-06T02:45:27.334582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334672+0000","last_change":"2023-10-06T02:45:22.804236+0000","last_active":"2023-10-06T02:45:27.334672+0000","last_peered":"2023-10-06T02:45:27.334672+0000","last_clean":"2023-10-06T02:45:27.334672+0000","last_became_active":"2023-10-06T02:45:22.802766+0000","last_became_peered":"2023-10-06T02:45:22.802766+0000","last_unstale":"2023-10-06T02:45:27.334672+0000","last_undegraded":"2023-10-06T02:45:27.334672+0000","last_fullsized":"2023-10-06T02:45:27.334672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341457+0000","last_change":"2023-10-06T02:45:23.324554+0000","last_active":"2023-10-06T02:45:27.341457+0000","last_peered":"2023-10-06T02:45:27.341457+0000","last_clean":"2023-10-06T02:45:27.341457+0000","last_became_active":"2023-10-06T02:45:23.324267+0000","last_became_peered":"2023-10-06T02:45:23.324267+0000","last_unstale":"2023-10-06T02:45:27.341457+0000","last_undegraded":"2023-10-06T02:45:27.341457+0000","last_fullsized":"2023-10-06T02:45:27.341457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354813+0000","last_change":"2023-10-06T02:45:27.346623+0000","last_active":"2023-10-06T02:45:27.354813+0000","last_peered":"2023-10-06T02:45:27.354813+0000","last_clean":"2023-10-06T02:45:27.354813+0000","last_became_active":"2023-10-06T02:45:27.346022+0000","last_became_peered":"2023-10-06T02:45:27.346022+0000","last_unstale":"2023-10-06T02:45:27.354813+0000","last_undegraded":"2023-10-06T02:45:27.354813+0000","last_fullsized":"2023-10-06T02:45:27.354813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:40.724648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332912+0000","last_change":"2023-10-06T02:45:22.805769+0000","last_active":"2023-10-06T02:45:27.332912+0000","last_peered":"2023-10-06T02:45:27.332912+0000","last_clean":"2023-10-06T02:45:27.332912+0000","last_became_active":"2023-10-06T02:45:22.805441+0000","last_became_peered":"2023-10-06T02:45:22.805441+0000","last_unstale":"2023-10-06T02:45:27.332912+0000","last_undegraded":"2023-10-06T02:45:27.332912+0000","last_fullsized":"2023-10-06T02:45:27.332912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:41.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.338579+0000","last_change":"2023-10-06T02:45:23.312634+0000","last_active":"2023-10-06T02:45:27.338579+0000","last_peered":"2023-10-06T02:45:27.338579+0000","last_clean":"2023-10-06T02:45:27.338579+0000","last_became_active":"2023-10-06T02:45:23.312292+0000","last_became_peered":"2023-10-06T02:45:23.312292+0000","last_unstale":"2023-10-06T02:45:27.338579+0000","last_undegraded":"2023-10-06T02:45:27.338579+0000","last_fullsized":"2023-10-06T02:45:27.338579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:05.795770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357104+0000","last_change":"2023-10-06T02:45:27.354353+0000","last_active":"2023-10-06T02:45:27.357104+0000","last_peered":"2023-10-06T02:45:27.357104+0000","last_clean":"2023-10-06T02:45:27.357104+0000","last_became_active":"2023-10-06T02:45:27.353428+0000","last_became_peered":"2023-10-06T02:45:27.353428+0000","last_unstale":"2023-10-06T02:45:27.357104+0000","last_undegraded":"2023-10-06T02:45:27.357104+0000","last_fullsized":"2023-10-06T02:45:27.357104+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.149006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325192+0000","last_change":"2023-10-06T02:45:23.309222+0000","last_active":"2023-10-06T02:45:27.325192+0000","last_peered":"2023-10-06T02:45:27.325192+0000","last_clean":"2023-10-06T02:45:27.325192+0000","last_became_active":"2023-10-06T02:45:23.308920+0000","last_became_peered":"2023-10-06T02:45:23.308920+0000","last_unstale":"2023-10-06T02:45:27.325192+0000","last_undegraded":"2023-10-06T02:45:27.325192+0000","last_fullsized":"2023-10-06T02:45:27.325192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:41.399143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329866+0000","last_change":"2023-10-06T02:45:23.312534+0000","last_active":"2023-10-06T02:45:27.329866+0000","last_peered":"2023-10-06T02:45:27.329866+0000","last_clean":"2023-10-06T02:45:27.329866+0000","last_became_active":"2023-10-06T02:45:23.312215+0000","last_became_peered":"2023-10-06T02:45:23.312215+0000","last_unstale":"2023-10-06T02:45:27.329866+0000","last_undegraded":"2023-10-06T02:45:27.329866+0000","last_fullsized":"2023-10-06T02:45:27.329866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:39.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336549+0000","last_change":"2023-10-06T02:45:23.305501+0000","last_active":"2023-10-06T02:45:27.336549+0000","last_peered":"2023-10-06T02:45:27.336549+0000","last_clean":"2023-10-06T02:45:27.336549+0000","last_became_active":"2023-10-06T02:45:23.305062+0000","last_became_peered":"2023-10-06T02:45:23.305062+0000","last_unstale":"2023-10-06T02:45:27.336549+0000","last_undegraded":"2023-10-06T02:45:27.336549+0000","last_fullsized":"2023-10-06T02:45:27.336549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333770+0000","last_change":"2023-10-06T02:45:23.324370+0000","last_active":"2023-10-06T02:45:27.333770+0000","last_peered":"2023-10-06T02:45:27.333770+0000","last_clean":"2023-10-06T02:45:27.333770+0000","last_became_active":"2023-10-06T02:45:23.324082+0000","last_became_peered":"2023-10-06T02:45:23.324082+0000","last_unstale":"2023-10-06T02:45:27.333770+0000","last_undegraded":"2023-10-06T02:45:27.333770+0000","last_fullsized":"2023-10-06T02:45:27.333770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:35.146269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337261+0000","last_change":"2023-10-06T02:45:23.322276+0000","last_active":"2023-10-06T02:45:27.337261+0000","last_peered":"2023-10-06T02:45:27.337261+0000","last_clean":"2023-10-06T02:45:27.337261+0000","last_became_active":"2023-10-06T02:45:23.321381+0000","last_became_peered":"2023-10-06T02:45:23.321381+0000","last_unstale":"2023-10-06T02:45:27.337261+0000","last_undegraded":"2023-10-06T02:45:27.337261+0000","last_fullsized":"2023-10-06T02:45:27.337261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341408+0000","last_change":"2023-10-06T02:45:23.324552+0000","last_active":"2023-10-06T02:45:27.341408+0000","last_peered":"2023-10-06T02:45:27.341408+0000","last_clean":"2023-10-06T02:45:27.341408+0000","last_became_active":"2023-10-06T02:45:23.324145+0000","last_became_peered":"2023-10-06T02:45:23.324145+0000","last_unstale":"2023-10-06T02:45:27.341408+0000","last_undegraded":"2023-10-06T02:45:27.341408+0000","last_fullsized":"2023-10-06T02:45:27.341408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340251+0000","last_change":"2023-10-06T02:45:23.316105+0000","last_active":"2023-10-06T02:45:27.340251+0000","last_peered":"2023-10-06T02:45:27.340251+0000","last_clean":"2023-10-06T02:45:27.340251+0000","last_became_active":"2023-10-06T02:45:23.315133+0000","last_became_peered":"2023-10-06T02:45:23.315133+0000","last_unstale":"2023-10-06T02:45:27.340251+0000","last_undegraded":"2023-10-06T02:45:27.340251+0000","last_fullsized":"2023-10-06T02:45:27.340251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.339997+0000","last_change":"2023-10-06T02:45:23.315937+0000","last_active":"2023-10-06T02:45:27.339997+0000","last_peered":"2023-10-06T02:45:27.339997+0000","last_clean":"2023-10-06T02:45:27.339997+0000","last_became_active":"2023-10-06T02:45:23.315541+0000","last_became_peered":"2023-10-06T02:45:23.315541+0000","last_unstale":"2023-10-06T02:45:27.339997+0000","last_undegraded":"2023-10-06T02:45:27.339997+0000","last_fullsized":"2023-10-06T02:45:27.339997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331530+0000","last_change":"2023-10-06T02:45:23.326576+0000","last_active":"2023-10-06T02:45:27.331530+0000","last_peered":"2023-10-06T02:45:27.331530+0000","last_clean":"2023-10-06T02:45:27.331530+0000","last_became_active":"2023-10-06T02:45:23.326298+0000","last_became_peered":"2023-10-06T02:45:23.326298+0000","last_unstale":"2023-10-06T02:45:27.331530+0000","last_undegraded":"2023-10-06T02:45:27.331530+0000","last_fullsized":"2023-10-06T02:45:27.331530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329766+0000","last_change":"2023-10-06T02:45:23.315079+0000","last_active":"2023-10-06T02:45:27.329766+0000","last_peered":"2023-10-06T02:45:27.329766+0000","last_clean":"2023-10-06T02:45:27.329766+0000","last_became_active":"2023-10-06T02:45:23.314780+0000","last_became_peered":"2023-10-06T02:45:23.314780+0000","last_unstale":"2023-10-06T02:45:27.329766+0000","last_undegraded":"2023-10-06T02:45:27.329766+0000","last_fullsized":"2023-10-06T02:45:27.329766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:22.869594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:23.314883+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:23.314594+0000","last_became_peered":"2023-10-06T02:45:23.314594+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332695+0000","last_change":"2023-10-06T02:45:23.324062+0000","last_active":"2023-10-06T02:45:27.332695+0000","last_peered":"2023-10-06T02:45:27.332695+0000","last_clean":"2023-10-06T02:45:27.332695+0000","last_became_active":"2023-10-06T02:45:23.323735+0000","last_became_peered":"2023-10-06T02:45:23.323735+0000","last_unstale":"2023-10-06T02:45:27.332695+0000","last_undegraded":"2023-10-06T02:45:27.332695+0000","last_fullsized":"2023-10-06T02:45:27.332695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336547+0000","last_change":"2023-10-06T02:45:23.306965+0000","last_active":"2023-10-06T02:45:27.336547+0000","last_peered":"2023-10-06T02:45:27.336547+0000","last_clean":"2023-10-06T02:45:27.336547+0000","last_became_active":"2023-10-06T02:45:23.306269+0000","last_became_peered":"2023-10-06T02:45:23.306269+0000","last_unstale":"2023-10-06T02:45:27.336547+0000","last_undegraded":"2023-10-06T02:45:27.336547+0000","last_fullsized":"2023-10-06T02:45:27.336547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320250+0000","last_change":"2023-10-06T02:45:23.305622+0000","last_active":"2023-10-06T02:45:27.320250+0000","last_peered":"2023-10-06T02:45:27.320250+0000","last_clean":"2023-10-06T02:45:27.320250+0000","last_became_active":"2023-10-06T02:45:23.305323+0000","last_became_peered":"2023-10-06T02:45:23.305323+0000","last_unstale":"2023-10-06T02:45:27.320250+0000","last_undegraded":"2023-10-06T02:45:27.320250+0000","last_fullsized":"2023-10-06T02:45:27.320250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:24.265950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334435+0000","last_change":"2023-10-06T02:45:23.311009+0000","last_active":"2023-10-06T02:45:27.334435+0000","last_peered":"2023-10-06T02:45:27.334435+0000","last_clean":"2023-10-06T02:45:27.334435+0000","last_became_active":"2023-10-06T02:45:23.310671+0000","last_became_peered":"2023-10-06T02:45:23.310671+0000","last_unstale":"2023-10-06T02:45:27.334435+0000","last_undegraded":"2023-10-06T02:45:27.334435+0000","last_fullsized":"2023-10-06T02:45:27.334435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:06.358484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334894+0000","last_change":"2023-10-06T02:45:23.312474+0000","last_active":"2023-10-06T02:45:27.334894+0000","last_peered":"2023-10-06T02:45:27.334894+0000","last_clean":"2023-10-06T02:45:27.334894+0000","last_became_active":"2023-10-06T02:45:23.312147+0000","last_became_peered":"2023-10-06T02:45:23.312147+0000","last_unstale":"2023-10-06T02:45:27.334894+0000","last_undegraded":"2023-10-06T02:45:27.334894+0000","last_fullsized":"2023-10-06T02:45:27.334894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330934+0000","last_change":"2023-10-06T02:45:23.314526+0000","last_active":"2023-10-06T02:45:27.330934+0000","last_peered":"2023-10-06T02:45:27.330934+0000","last_clean":"2023-10-06T02:45:27.330934+0000","last_became_active":"2023-10-06T02:45:23.314229+0000","last_became_peered":"2023-10-06T02:45:23.314229+0000","last_unstale":"2023-10-06T02:45:27.330934+0000","last_undegraded":"2023-10-06T02:45:27.330934+0000","last_fullsized":"2023-10-06T02:45:27.330934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:08.831291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332089+0000","last_change":"2023-10-06T02:45:23.326220+0000","last_active":"2023-10-06T02:45:27.332089+0000","last_peered":"2023-10-06T02:45:27.332089+0000","last_clean":"2023-10-06T02:45:27.332089+0000","last_became_active":"2023-10-06T02:45:23.325853+0000","last_became_peered":"2023-10-06T02:45:23.325853+0000","last_unstale":"2023-10-06T02:45:27.332089+0000","last_undegraded":"2023-10-06T02:45:27.332089+0000","last_fullsized":"2023-10-06T02:45:27.332089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340313+0000","last_change":"2023-10-06T02:45:23.319007+0000","last_active":"2023-10-06T02:45:27.340313+0000","last_peered":"2023-10-06T02:45:27.340313+0000","last_clean":"2023-10-06T02:45:27.340313+0000","last_became_active":"2023-10-06T02:45:23.318521+0000","last_became_peered":"2023-10-06T02:45:23.318521+0000","last_unstale":"2023-10-06T02:45:27.340313+0000","last_undegraded":"2023-10-06T02:45:27.340313+0000","last_fullsized":"2023-10-06T02:45:27.340313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319562+0000","last_change":"2023-10-06T02:45:23.308224+0000","last_active":"2023-10-06T02:45:27.319562+0000","last_peered":"2023-10-06T02:45:27.319562+0000","last_clean":"2023-10-06T02:45:27.319562+0000","last_became_active":"2023-10-06T02:45:23.307908+0000","last_became_peered":"2023-10-06T02:45:23.307908+0000","last_unstale":"2023-10-06T02:45:27.319562+0000","last_undegraded":"2023-10-06T02:45:27.319562+0000","last_fullsized":"2023-10-06T02:45:27.319562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:26.518844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319631+0000","last_change":"2023-10-06T02:45:23.312138+0000","last_active":"2023-10-06T02:45:27.319631+0000","last_peered":"2023-10-06T02:45:27.319631+0000","last_clean":"2023-10-06T02:45:27.319631+0000","last_became_active":"2023-10-06T02:45:23.311612+0000","last_became_peered":"2023-10-06T02:45:23.311612+0000","last_unstale":"2023-10-06T02:45:27.319631+0000","last_undegraded":"2023-10-06T02:45:27.319631+0000","last_fullsized":"2023-10-06T02:45:27.319631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.807177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336407+0000","last_change":"2023-10-06T02:45:23.314057+0000","last_active":"2023-10-06T02:45:27.336407+0000","last_peered":"2023-10-06T02:45:27.336407+0000","last_clean":"2023-10-06T02:45:27.336407+0000","last_became_active":"2023-10-06T02:45:23.313762+0000","last_became_peered":"2023-10-06T02:45:23.313762+0000","last_unstale":"2023-10-06T02:45:27.336407+0000","last_undegraded":"2023-10-06T02:45:27.336407+0000","last_fullsized":"2023-10-06T02:45:27.336407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:54.392205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341622+0000","last_change":"2023-10-06T02:45:23.318996+0000","last_active":"2023-10-06T02:45:27.341622+0000","last_peered":"2023-10-06T02:45:27.341622+0000","last_clean":"2023-10-06T02:45:27.341622+0000","last_became_active":"2023-10-06T02:45:23.318710+0000","last_became_peered":"2023-10-06T02:45:23.318710+0000","last_unstale":"2023-10-06T02:45:27.341622+0000","last_undegraded":"2023-10-06T02:45:27.341622+0000","last_fullsized":"2023-10-06T02:45:27.341622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:04.789619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334795+0000","last_change":"2023-10-06T02:45:23.312737+0000","last_active":"2023-10-06T02:45:27.334795+0000","last_peered":"2023-10-06T02:45:27.334795+0000","last_clean":"2023-10-06T02:45:27.334795+0000","last_became_active":"2023-10-06T02:45:23.312446+0000","last_became_peered":"2023-10-06T02:45:23.312446+0000","last_unstale":"2023-10-06T02:45:27.334795+0000","last_undegraded":"2023-10-06T02:45:27.334795+0000","last_fullsized":"2023-10-06T02:45:27.334795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340404+0000","last_change":"2023-10-06T02:45:23.318887+0000","last_active":"2023-10-06T02:45:27.340404+0000","last_peered":"2023-10-06T02:45:27.340404+0000","last_clean":"2023-10-06T02:45:27.340404+0000","last_became_active":"2023-10-06T02:45:23.318573+0000","last_became_peered":"2023-10-06T02:45:23.318573+0000","last_unstale":"2023-10-06T02:45:27.340404+0000","last_undegraded":"2023-10-06T02:45:27.340404+0000","last_fullsized":"2023-10-06T02:45:27.340404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331986+0000","last_change":"2023-10-06T02:45:23.326294+0000","last_active":"2023-10-06T02:45:27.331986+0000","last_peered":"2023-10-06T02:45:27.331986+0000","last_clean":"2023-10-06T02:45:27.331986+0000","last_became_active":"2023-10-06T02:45:23.325935+0000","last_became_peered":"2023-10-06T02:45:23.325935+0000","last_unstale":"2023-10-06T02:45:27.331986+0000","last_undegraded":"2023-10-06T02:45:27.331986+0000","last_fullsized":"2023-10-06T02:45:27.331986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336417+0000","last_change":"2023-10-06T02:45:23.316621+0000","last_active":"2023-10-06T02:45:27.336417+0000","last_peered":"2023-10-06T02:45:27.336417+0000","last_clean":"2023-10-06T02:45:27.336417+0000","last_became_active":"2023-10-06T02:45:23.316283+0000","last_became_peered":"2023-10-06T02:45:23.316283+0000","last_unstale":"2023-10-06T02:45:27.336417+0000","last_undegraded":"2023-10-06T02:45:27.336417+0000","last_fullsized":"2023-10-06T02:45:27.336417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:21.187543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335121+0000","last_change":"2023-10-06T02:45:23.318488+0000","last_active":"2023-10-06T02:45:27.335121+0000","last_peered":"2023-10-06T02:45:27.335121+0000","last_clean":"2023-10-06T02:45:27.335121+0000","last_became_active":"2023-10-06T02:45:23.318144+0000","last_became_peered":"2023-10-06T02:45:23.318144+0000","last_unstale":"2023-10-06T02:45:27.335121+0000","last_undegraded":"2023-10-06T02:45:27.335121+0000","last_fullsized":"2023-10-06T02:45:27.335121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325094+0000","last_change":"2023-10-06T02:45:23.316019+0000","last_active":"2023-10-06T02:45:27.325094+0000","last_peered":"2023-10-06T02:45:27.325094+0000","last_clean":"2023-10-06T02:45:27.325094+0000","last_became_active":"2023-10-06T02:45:23.315706+0000","last_became_peered":"2023-10-06T02:45:23.315706+0000","last_unstale":"2023-10-06T02:45:27.325094+0000","last_undegraded":"2023-10-06T02:45:27.325094+0000","last_fullsized":"2023-10-06T02:45:27.325094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.498422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336118+0000","last_change":"2023-10-06T02:45:23.313938+0000","last_active":"2023-10-06T02:45:27.336118+0000","last_peered":"2023-10-06T02:45:27.336118+0000","last_clean":"2023-10-06T02:45:27.336118+0000","last_became_active":"2023-10-06T02:45:23.313340+0000","last_became_peered":"2023-10-06T02:45:23.313340+0000","last_unstale":"2023-10-06T02:45:27.336118+0000","last_undegraded":"2023-10-06T02:45:27.336118+0000","last_fullsized":"2023-10-06T02:45:27.336118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:53.828654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341529+0000","last_change":"2023-10-06T02:45:23.321544+0000","last_active":"2023-10-06T02:45:27.341529+0000","last_peered":"2023-10-06T02:45:27.341529+0000","last_clean":"2023-10-06T02:45:27.341529+0000","last_became_active":"2023-10-06T02:45:23.321216+0000","last_became_peered":"2023-10-06T02:45:23.321216+0000","last_unstale":"2023-10-06T02:45:27.341529+0000","last_undegraded":"2023-10-06T02:45:27.341529+0000","last_fullsized":"2023-10-06T02:45:27.341529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320111+0000","last_change":"2023-10-06T02:45:23.309006+0000","last_active":"2023-10-06T02:45:27.320111+0000","last_peered":"2023-10-06T02:45:27.320111+0000","last_clean":"2023-10-06T02:45:27.320111+0000","last_became_active":"2023-10-06T02:45:23.308617+0000","last_became_peered":"2023-10-06T02:45:23.308617+0000","last_unstale":"2023-10-06T02:45:27.320111+0000","last_undegraded":"2023-10-06T02:45:27.320111+0000","last_fullsized":"2023-10-06T02:45:27.320111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:35.295610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":131211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":315392,"data_stored":127531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":718363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:45:42.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:45:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.529+0000 7f753b59e700 1 Processor -- start 2023-10-06T02:45:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.529+0000 7f753b59e700 1 -- start start 2023-10-06T02:45:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753b59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f753c0abb50 0x7f753c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753b59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f753c155380 0x7f753c155820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753b59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c15a390 con 0x7f753c06c010 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753b59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c159570 con 0x7f753c0abb50 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753a59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f753c06c010 0x7f753c069540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58170/0 (socket says 172.21.15.33:58170) 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753a59c700 1 -- 172.21.15.33:0/4034271505 learned_addr learned my addr 172.21.15.33:0/4034271505 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753b59e700 1 -- 172.21.15.33:0/4034271505 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c159780 con 0x7f753c155380 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7533fff700 1 --2- 172.21.15.33:0/4034271505 >> v2:172.21.15.33:3300/0 conn(0x7f753c0abb50 0x7f753c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f753ad9d700 1 --2- 172.21.15.33:0/4034271505 >> v2:172.21.15.167:3300/0 conn(0x7f753c155380 0x7f753c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7539d9b700 1 -- 172.21.15.33:0/4034271505 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548433413 0 0) 0x7f753c15a390 con 0x7f753c06c010 2023-10-06T02:45:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7539d9b700 1 -- 172.21.15.33:0/4034271505 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7528003570 con 0x7f753c06c010 2023-10-06T02:45:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7539d9b700 1 -- 172.21.15.33:0/4034271505 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1480415867 0 0) 0x7f7528003570 con 0x7f753c06c010 2023-10-06T02:45:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7533fff700 1 -- 172.21.15.33:0/4034271505 >> v2:172.21.15.167:3300/0 conn(0x7f753c155380 msgr2=0x7f753c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7533fff700 1 --2- 172.21.15.33:0/4034271505 >> v2:172.21.15.167:3300/0 conn(0x7f753c155380 0x7f753c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7533fff700 1 -- 172.21.15.33:0/4034271505 >> v1:172.21.15.33:6789/0 conn(0x7f753c06c010 legacy=0x7f753c069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7533fff700 1 -- 172.21.15.33:0/4034271505 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c159a60 con 0x7f753c0abb50 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7533fff700 1 --2- 172.21.15.33:0/4034271505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c0abb50 0x7f753c06c3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75240098e0 tx=0x7f75240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c976fc989faa6cb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7539d9b700 1 -- 172.21.15.33:0/4034271505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c0015f0 con 0x7f753c0abb50 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7539d9b700 1 -- 172.21.15.33:0/4034271505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7524011420 con 0x7f753c0abb50 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.533+0000 7f7539d9b700 1 -- 172.21.15.33:0/4034271505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240117c0 con 0x7f753c0abb50 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 -- 172.21.15.33:0/4034271505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c0abb50 msgr2=0x7f753c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 --2- 172.21.15.33:0/4034271505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c0abb50 0x7f753c06c3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75240098e0 tx=0x7f75240091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 -- 172.21.15.33:0/4034271505 shutdown_connections 2023-10-06T02:45:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 --2- 172.21.15.33:0/4034271505 >> v2:172.21.15.167:3300/0 conn(0x7f753c155380 0x7f753c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 --2- 172.21.15.33:0/4034271505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c0abb50 0x7f753c06c3e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 -- 172.21.15.33:0/4034271505 >> 172.21.15.33:0/4034271505 conn(0x7f753c06e6c0 msgr2=0x7f753c06ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:42.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 -- 172.21.15.33:0/4034271505 shutdown_connections 2023-10-06T02:45:42.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 -- 172.21.15.33:0/4034271505 wait complete. 2023-10-06T02:45:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 Processor -- start 2023-10-06T02:45:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.537+0000 7f753b59e700 1 -- start start 2023-10-06T02:45:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 0x7f753c138710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c0abb50 0x7f753c14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f753c155380 0x7f753c14c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c14cb30 con 0x7f753c06c010 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c14e4a0 con 0x7f753c0abb50 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c14d920 con 0x7f753c155380 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 0x7f753c138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 0x7f753c138710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56628/0 (socket says 172.21.15.33:56628) 2023-10-06T02:45:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 -- 172.21.15.33:0/50866962 learned_addr learned my addr 172.21.15.33:0/50866962 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753ad9d700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f753c155380 0x7f753c14c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f7533fff700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c0abb50 0x7f753c14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 -- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f753c155380 msgr2=0x7f753c14c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f753c155380 0x7f753c14c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 -- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c0abb50 msgr2=0x7f753c14c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c0abb50 0x7f753c14c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 -- 172.21.15.33:0/50866962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009420 con 0x7f753c06c010 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f7533fff700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c0abb50 0x7f753c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753a59c700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 0x7f753c138710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f752c00cb60 tx=0x7f752c001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c0073d0 con 0x7f753c06c010 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c007550 con 0x7f753c06c010 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c14cd10 con 0x7f753c06c010 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c005a40 con 0x7f753c06c010 2023-10-06T02:45:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.541+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c14d2d0 con 0x7f753c06c010 2023-10-06T02:45:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.545+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c071040 con 0x7f753c06c010 2023-10-06T02:45:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.545+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f752c00a550 con 0x7f753c06c010 2023-10-06T02:45:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.545+0000 7f75327fc700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f751c060550 0x7f751c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.545+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f752c07cbf0 con 0x7f753c06c010 2023-10-06T02:45:42.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.545+0000 7f7533fff700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f751c060550 0x7f751c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.545+0000 7f7533fff700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f751c060550 0x7f751c062a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f753c0abf40 tx=0x7f7524016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.553+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f752c04cd70 con 0x7f753c06c010 2023-10-06T02:45:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.713+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f753c159d40 con 0x7f751c060550 2023-10-06T02:45:42.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.733+0000 7f75327fc700 1 -- 172.21.15.33:0/50866962 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367276 (secure 0 0 0) 0x7f753c159d40 con 0x7f751c060550 2023-10-06T02:45:42.745 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f751c060550 msgr2=0x7f751c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f751c060550 0x7f751c062a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f753c0abf40 tx=0x7f7524016000 comp rx=0 tx=0).stop 2023-10-06T02:45:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 msgr2=0x7f753c138710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 0x7f753c138710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f752c00cb60 tx=0x7f752c001440 comp rx=0 tx=0).stop 2023-10-06T02:45:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 shutdown_connections 2023-10-06T02:45:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f751c060550 0x7f751c062a00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f753c155380 0x7f753c14c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c0abb50 0x7f753c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 --2- 172.21.15.33:0/50866962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753c06c010 0x7f753c138710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 >> 172.21.15.33:0/50866962 conn(0x7f753c06e6c0 msgr2=0x7f753c1572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:42.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 shutdown_connections 2023-10-06T02:45:42.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.741+0000 7f753b59e700 1 -- 172.21.15.33:0/50866962 wait complete. 2023-10-06T02:45:42.750 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:45:42.761 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-06T02:45:42.565364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381960,"kb_used_data":3656,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592760,"statfs":{"total":773094113280,"available":770654986240,"internally_reserved":0,"allocated":3743744,"data_stored":2209232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005418"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.352797+0000","last_change":"2023-10-06T02:45:27.343811+0000","last_active":"2023-10-06T02:45:27.352797+0000","last_peered":"2023-10-06T02:45:27.352797+0000","last_clean":"2023-10-06T02:45:27.352797+0000","last_became_active":"2023-10-06T02:45:27.343175+0000","last_became_peered":"2023-10-06T02:45:27.343175+0000","last_unstale":"2023-10-06T02:45:27.352797+0000","last_undegraded":"2023-10-06T02:45:27.352797+0000","last_fullsized":"2023-10-06T02:45:27.352797+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334994+0000","last_change":"2023-10-06T02:45:22.801080+0000","last_active":"2023-10-06T02:45:27.334994+0000","last_peered":"2023-10-06T02:45:27.334994+0000","last_clean":"2023-10-06T02:45:27.334994+0000","last_became_active":"2023-10-06T02:45:22.800763+0000","last_became_peered":"2023-10-06T02:45:22.800763+0000","last_unstale":"2023-10-06T02:45:27.334994+0000","last_undegraded":"2023-10-06T02:45:27.334994+0000","last_fullsized":"2023-10-06T02:45:27.334994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333783+0000","last_change":"2023-10-06T02:45:23.305500+0000","last_active":"2023-10-06T02:45:27.333783+0000","last_peered":"2023-10-06T02:45:27.333783+0000","last_clean":"2023-10-06T02:45:27.333783+0000","last_became_active":"2023-10-06T02:45:23.305199+0000","last_became_peered":"2023-10-06T02:45:23.305199+0000","last_unstale":"2023-10-06T02:45:27.333783+0000","last_undegraded":"2023-10-06T02:45:27.333783+0000","last_fullsized":"2023-10-06T02:45:27.333783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351635+0000","last_change":"2023-10-06T02:45:27.345953+0000","last_active":"2023-10-06T02:45:27.351635+0000","last_peered":"2023-10-06T02:45:27.351635+0000","last_clean":"2023-10-06T02:45:27.351635+0000","last_became_active":"2023-10-06T02:45:27.345526+0000","last_became_peered":"2023-10-06T02:45:27.345526+0000","last_unstale":"2023-10-06T02:45:27.351635+0000","last_undegraded":"2023-10-06T02:45:27.351635+0000","last_fullsized":"2023-10-06T02:45:27.351635+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:37.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332907+0000","last_change":"2023-10-06T02:45:22.805277+0000","last_active":"2023-10-06T02:45:27.332907+0000","last_peered":"2023-10-06T02:45:27.332907+0000","last_clean":"2023-10-06T02:45:27.332907+0000","last_became_active":"2023-10-06T02:45:22.804802+0000","last_became_peered":"2023-10-06T02:45:22.804802+0000","last_unstale":"2023-10-06T02:45:27.332907+0000","last_undegraded":"2023-10-06T02:45:27.332907+0000","last_fullsized":"2023-10-06T02:45:27.332907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:37.831567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330139+0000","last_change":"2023-10-06T02:45:23.306937+0000","last_active":"2023-10-06T02:45:27.330139+0000","last_peered":"2023-10-06T02:45:27.330139+0000","last_clean":"2023-10-06T02:45:27.330139+0000","last_became_active":"2023-10-06T02:45:23.306641+0000","last_became_peered":"2023-10-06T02:45:23.306641+0000","last_unstale":"2023-10-06T02:45:27.330139+0000","last_undegraded":"2023-10-06T02:45:27.330139+0000","last_fullsized":"2023-10-06T02:45:27.330139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:23.162123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356992+0000","last_change":"2023-10-06T02:45:27.353931+0000","last_active":"2023-10-06T02:45:27.356992+0000","last_peered":"2023-10-06T02:45:27.356992+0000","last_clean":"2023-10-06T02:45:27.356992+0000","last_became_active":"2023-10-06T02:45:27.353488+0000","last_became_peered":"2023-10-06T02:45:27.353488+0000","last_unstale":"2023-10-06T02:45:27.356992+0000","last_undegraded":"2023-10-06T02:45:27.356992+0000","last_fullsized":"2023-10-06T02:45:27.356992+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341505+0000","last_change":"2023-10-06T02:45:22.803357+0000","last_active":"2023-10-06T02:45:27.341505+0000","last_peered":"2023-10-06T02:45:27.341505+0000","last_clean":"2023-10-06T02:45:27.341505+0000","last_became_active":"2023-10-06T02:45:22.800673+0000","last_became_peered":"2023-10-06T02:45:22.800673+0000","last_unstale":"2023-10-06T02:45:27.341505+0000","last_undegraded":"2023-10-06T02:45:27.341505+0000","last_fullsized":"2023-10-06T02:45:27.341505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336470+0000","last_change":"2023-10-06T02:45:23.310269+0000","last_active":"2023-10-06T02:45:27.336470+0000","last_peered":"2023-10-06T02:45:27.336470+0000","last_clean":"2023-10-06T02:45:27.336470+0000","last_became_active":"2023-10-06T02:45:23.309927+0000","last_became_peered":"2023-10-06T02:45:23.309927+0000","last_unstale":"2023-10-06T02:45:27.336470+0000","last_undegraded":"2023-10-06T02:45:27.336470+0000","last_fullsized":"2023-10-06T02:45:27.336470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351290+0000","last_change":"2023-10-06T02:45:27.348258+0000","last_active":"2023-10-06T02:45:27.351290+0000","last_peered":"2023-10-06T02:45:27.351290+0000","last_clean":"2023-10-06T02:45:27.351290+0000","last_became_active":"2023-10-06T02:45:27.347848+0000","last_became_peered":"2023-10-06T02:45:27.347848+0000","last_unstale":"2023-10-06T02:45:27.351290+0000","last_undegraded":"2023-10-06T02:45:27.351290+0000","last_fullsized":"2023-10-06T02:45:27.351290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340794+0000","last_change":"2023-10-06T02:45:22.800086+0000","last_active":"2023-10-06T02:45:27.340794+0000","last_peered":"2023-10-06T02:45:27.340794+0000","last_clean":"2023-10-06T02:45:27.340794+0000","last_became_active":"2023-10-06T02:45:22.799739+0000","last_became_peered":"2023-10-06T02:45:22.799739+0000","last_unstale":"2023-10-06T02:45:27.340794+0000","last_undegraded":"2023-10-06T02:45:27.340794+0000","last_fullsized":"2023-10-06T02:45:27.340794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332497+0000","last_change":"2023-10-06T02:45:23.312373+0000","last_active":"2023-10-06T02:45:27.332497+0000","last_peered":"2023-10-06T02:45:27.332497+0000","last_clean":"2023-10-06T02:45:27.332497+0000","last_became_active":"2023-10-06T02:45:23.311513+0000","last_became_peered":"2023-10-06T02:45:23.311513+0000","last_unstale":"2023-10-06T02:45:27.332497+0000","last_undegraded":"2023-10-06T02:45:27.332497+0000","last_fullsized":"2023-10-06T02:45:27.332497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:10.715925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360984+0000","last_change":"2023-10-06T02:45:27.347884+0000","last_active":"2023-10-06T02:45:27.360984+0000","last_peered":"2023-10-06T02:45:27.360984+0000","last_clean":"2023-10-06T02:45:27.360984+0000","last_became_active":"2023-10-06T02:45:27.347285+0000","last_became_peered":"2023-10-06T02:45:27.347285+0000","last_unstale":"2023-10-06T02:45:27.360984+0000","last_undegraded":"2023-10-06T02:45:27.360984+0000","last_fullsized":"2023-10-06T02:45:27.360984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:52.629371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335221+0000","last_change":"2023-10-06T02:45:22.804322+0000","last_active":"2023-10-06T02:45:27.335221+0000","last_peered":"2023-10-06T02:45:27.335221+0000","last_clean":"2023-10-06T02:45:27.335221+0000","last_became_active":"2023-10-06T02:45:22.802485+0000","last_became_peered":"2023-10-06T02:45:22.802485+0000","last_unstale":"2023-10-06T02:45:27.335221+0000","last_undegraded":"2023-10-06T02:45:27.335221+0000","last_fullsized":"2023-10-06T02:45:27.335221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320001+0000","last_change":"2023-10-06T02:45:23.311929+0000","last_active":"2023-10-06T02:45:27.320001+0000","last_peered":"2023-10-06T02:45:27.320001+0000","last_clean":"2023-10-06T02:45:27.320001+0000","last_became_active":"2023-10-06T02:45:23.311209+0000","last_became_peered":"2023-10-06T02:45:23.311209+0000","last_unstale":"2023-10-06T02:45:27.320001+0000","last_undegraded":"2023-10-06T02:45:27.320001+0000","last_fullsized":"2023-10-06T02:45:27.320001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:39.089521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354213+0000","last_change":"2023-10-06T02:45:27.347958+0000","last_active":"2023-10-06T02:45:27.354213+0000","last_peered":"2023-10-06T02:45:27.354213+0000","last_clean":"2023-10-06T02:45:27.354213+0000","last_became_active":"2023-10-06T02:45:27.347490+0000","last_became_peered":"2023-10-06T02:45:27.347490+0000","last_unstale":"2023-10-06T02:45:27.354213+0000","last_undegraded":"2023-10-06T02:45:27.354213+0000","last_fullsized":"2023-10-06T02:45:27.354213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:40.393467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336567+0000","last_change":"2023-10-06T02:45:22.799339+0000","last_active":"2023-10-06T02:45:27.336567+0000","last_peered":"2023-10-06T02:45:27.336567+0000","last_clean":"2023-10-06T02:45:27.336567+0000","last_became_active":"2023-10-06T02:45:22.798982+0000","last_became_peered":"2023-10-06T02:45:22.798982+0000","last_unstale":"2023-10-06T02:45:27.336567+0000","last_undegraded":"2023-10-06T02:45:27.336567+0000","last_fullsized":"2023-10-06T02:45:27.336567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:41.206862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330449+0000","last_change":"2023-10-06T02:45:23.312269+0000","last_active":"2023-10-06T02:45:27.330449+0000","last_peered":"2023-10-06T02:45:27.330449+0000","last_clean":"2023-10-06T02:45:27.330449+0000","last_became_active":"2023-10-06T02:45:23.311975+0000","last_became_peered":"2023-10-06T02:45:23.311975+0000","last_unstale":"2023-10-06T02:45:27.330449+0000","last_undegraded":"2023-10-06T02:45:27.330449+0000","last_fullsized":"2023-10-06T02:45:27.330449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:27.851854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354212+0000","last_change":"2023-10-06T02:45:27.347553+0000","last_active":"2023-10-06T02:45:27.354212+0000","last_peered":"2023-10-06T02:45:27.354212+0000","last_clean":"2023-10-06T02:45:27.354212+0000","last_became_active":"2023-10-06T02:45:27.347165+0000","last_became_peered":"2023-10-06T02:45:27.347165+0000","last_unstale":"2023-10-06T02:45:27.354212+0000","last_undegraded":"2023-10-06T02:45:27.354212+0000","last_fullsized":"2023-10-06T02:45:27.354212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:38.681094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:22.808457+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:22.808145+0000","last_became_peered":"2023-10-06T02:45:22.808145+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334405+0000","last_change":"2023-10-06T02:45:23.323552+0000","last_active":"2023-10-06T02:45:27.334405+0000","last_peered":"2023-10-06T02:45:27.334405+0000","last_clean":"2023-10-06T02:45:27.334405+0000","last_became_active":"2023-10-06T02:45:23.323241+0000","last_became_peered":"2023-10-06T02:45:23.323241+0000","last_unstale":"2023-10-06T02:45:27.334405+0000","last_undegraded":"2023-10-06T02:45:27.334405+0000","last_fullsized":"2023-10-06T02:45:27.334405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353329+0000","last_change":"2023-10-06T02:45:27.349343+0000","last_active":"2023-10-06T02:45:27.353329+0000","last_peered":"2023-10-06T02:45:27.353329+0000","last_clean":"2023-10-06T02:45:27.353329+0000","last_became_active":"2023-10-06T02:45:27.348859+0000","last_became_peered":"2023-10-06T02:45:27.348859+0000","last_unstale":"2023-10-06T02:45:27.353329+0000","last_undegraded":"2023-10-06T02:45:27.353329+0000","last_fullsized":"2023-10-06T02:45:27.353329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333023+0000","last_change":"2023-10-06T02:45:22.811866+0000","last_active":"2023-10-06T02:45:27.333023+0000","last_peered":"2023-10-06T02:45:27.333023+0000","last_clean":"2023-10-06T02:45:27.333023+0000","last_became_active":"2023-10-06T02:45:22.811334+0000","last_became_peered":"2023-10-06T02:45:22.811334+0000","last_unstale":"2023-10-06T02:45:27.333023+0000","last_undegraded":"2023-10-06T02:45:27.333023+0000","last_fullsized":"2023-10-06T02:45:27.333023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325429+0000","last_change":"2023-10-06T02:45:23.320977+0000","last_active":"2023-10-06T02:45:27.325429+0000","last_peered":"2023-10-06T02:45:27.325429+0000","last_clean":"2023-10-06T02:45:27.325429+0000","last_became_active":"2023-10-06T02:45:23.320677+0000","last_became_peered":"2023-10-06T02:45:23.320677+0000","last_unstale":"2023-10-06T02:45:27.325429+0000","last_undegraded":"2023-10-06T02:45:27.325429+0000","last_fullsized":"2023-10-06T02:45:27.325429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:18.520846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332892+0000","last_change":"2023-10-06T02:45:23.312712+0000","last_active":"2023-10-06T02:45:27.332892+0000","last_peered":"2023-10-06T02:45:27.332892+0000","last_clean":"2023-10-06T02:45:27.332892+0000","last_became_active":"2023-10-06T02:45:23.312164+0000","last_became_peered":"2023-10-06T02:45:23.312164+0000","last_unstale":"2023-10-06T02:45:27.332892+0000","last_undegraded":"2023-10-06T02:45:27.332892+0000","last_fullsized":"2023-10-06T02:45:27.332892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.186960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335089+0000","last_change":"2023-10-06T02:45:22.800781+0000","last_active":"2023-10-06T02:45:27.335089+0000","last_peered":"2023-10-06T02:45:27.335089+0000","last_clean":"2023-10-06T02:45:27.335089+0000","last_became_active":"2023-10-06T02:45:22.800484+0000","last_became_peered":"2023-10-06T02:45:22.800484+0000","last_unstale":"2023-10-06T02:45:27.335089+0000","last_undegraded":"2023-10-06T02:45:27.335089+0000","last_fullsized":"2023-10-06T02:45:27.335089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354176+0000","last_change":"2023-10-06T02:45:27.348699+0000","last_active":"2023-10-06T02:45:27.354176+0000","last_peered":"2023-10-06T02:45:27.354176+0000","last_clean":"2023-10-06T02:45:27.354176+0000","last_became_active":"2023-10-06T02:45:27.348111+0000","last_became_peered":"2023-10-06T02:45:27.348111+0000","last_unstale":"2023-10-06T02:45:27.354176+0000","last_undegraded":"2023-10-06T02:45:27.354176+0000","last_fullsized":"2023-10-06T02:45:27.354176+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:26.404420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330041+0000","last_change":"2023-10-06T02:45:23.323471+0000","last_active":"2023-10-06T02:45:27.330041+0000","last_peered":"2023-10-06T02:45:27.330041+0000","last_clean":"2023-10-06T02:45:27.330041+0000","last_became_active":"2023-10-06T02:45:23.323203+0000","last_became_peered":"2023-10-06T02:45:23.323203+0000","last_unstale":"2023-10-06T02:45:27.330041+0000","last_undegraded":"2023-10-06T02:45:27.330041+0000","last_fullsized":"2023-10-06T02:45:27.330041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:59.446720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341624+0000","last_change":"2023-10-06T02:45:22.804313+0000","last_active":"2023-10-06T02:45:27.341624+0000","last_peered":"2023-10-06T02:45:27.341624+0000","last_clean":"2023-10-06T02:45:27.341624+0000","last_became_active":"2023-10-06T02:45:22.804008+0000","last_became_peered":"2023-10-06T02:45:22.804008+0000","last_unstale":"2023-10-06T02:45:27.341624+0000","last_undegraded":"2023-10-06T02:45:27.341624+0000","last_fullsized":"2023-10-06T02:45:27.341624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356829+0000","last_change":"2023-10-06T02:45:27.355277+0000","last_active":"2023-10-06T02:45:27.356829+0000","last_peered":"2023-10-06T02:45:27.356829+0000","last_clean":"2023-10-06T02:45:27.356829+0000","last_became_active":"2023-10-06T02:45:27.354885+0000","last_became_peered":"2023-10-06T02:45:27.354885+0000","last_unstale":"2023-10-06T02:45:27.356829+0000","last_undegraded":"2023-10-06T02:45:27.356829+0000","last_fullsized":"2023-10-06T02:45:27.356829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.722369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333001+0000","last_change":"2023-10-06T02:45:22.810372+0000","last_active":"2023-10-06T02:45:27.333001+0000","last_peered":"2023-10-06T02:45:27.333001+0000","last_clean":"2023-10-06T02:45:27.333001+0000","last_became_active":"2023-10-06T02:45:22.810117+0000","last_became_peered":"2023-10-06T02:45:22.810117+0000","last_unstale":"2023-10-06T02:45:27.333001+0000","last_undegraded":"2023-10-06T02:45:27.333001+0000","last_fullsized":"2023-10-06T02:45:27.333001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:54.333503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335943+0000","last_change":"2023-10-06T02:45:23.319583+0000","last_active":"2023-10-06T02:45:27.335943+0000","last_peered":"2023-10-06T02:45:27.335943+0000","last_clean":"2023-10-06T02:45:27.335943+0000","last_became_active":"2023-10-06T02:45:23.319266+0000","last_became_peered":"2023-10-06T02:45:23.319266+0000","last_unstale":"2023-10-06T02:45:27.335943+0000","last_undegraded":"2023-10-06T02:45:27.335943+0000","last_fullsized":"2023-10-06T02:45:27.335943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353966+0000","last_change":"2023-10-06T02:45:27.351069+0000","last_active":"2023-10-06T02:45:27.353966+0000","last_peered":"2023-10-06T02:45:27.353966+0000","last_clean":"2023-10-06T02:45:27.353966+0000","last_became_active":"2023-10-06T02:45:27.350579+0000","last_became_peered":"2023-10-06T02:45:27.350579+0000","last_unstale":"2023-10-06T02:45:27.353966+0000","last_undegraded":"2023-10-06T02:45:27.353966+0000","last_fullsized":"2023-10-06T02:45:27.353966+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:47.948820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"29'9","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:33.113646+0000","last_change":"2023-10-06T02:45:22.804024+0000","last_active":"2023-10-06T02:45:33.113646+0000","last_peered":"2023-10-06T02:45:33.113646+0000","last_clean":"2023-10-06T02:45:33.113646+0000","last_became_active":"2023-10-06T02:45:22.803679+0000","last_became_peered":"2023-10-06T02:45:22.803679+0000","last_unstale":"2023-10-06T02:45:33.113646+0000","last_undegraded":"2023-10-06T02:45:33.113646+0000","last_fullsized":"2023-10-06T02:45:33.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.507771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332592+0000","last_change":"2023-10-06T02:45:23.312638+0000","last_active":"2023-10-06T02:45:27.332592+0000","last_peered":"2023-10-06T02:45:27.332592+0000","last_clean":"2023-10-06T02:45:27.332592+0000","last_became_active":"2023-10-06T02:45:23.312006+0000","last_became_peered":"2023-10-06T02:45:23.312006+0000","last_unstale":"2023-10-06T02:45:27.332592+0000","last_undegraded":"2023-10-06T02:45:27.332592+0000","last_fullsized":"2023-10-06T02:45:27.332592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.358568+0000","last_change":"2023-10-06T02:45:27.345474+0000","last_active":"2023-10-06T02:45:27.358568+0000","last_peered":"2023-10-06T02:45:27.358568+0000","last_clean":"2023-10-06T02:45:27.358568+0000","last_became_active":"2023-10-06T02:45:27.345014+0000","last_became_peered":"2023-10-06T02:45:27.345014+0000","last_unstale":"2023-10-06T02:45:27.358568+0000","last_undegraded":"2023-10-06T02:45:27.358568+0000","last_fullsized":"2023-10-06T02:45:27.358568+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334989+0000","last_change":"2023-10-06T02:45:23.319302+0000","last_active":"2023-10-06T02:45:27.334989+0000","last_peered":"2023-10-06T02:45:27.334989+0000","last_clean":"2023-10-06T02:45:27.334989+0000","last_became_active":"2023-10-06T02:45:23.318951+0000","last_became_peered":"2023-10-06T02:45:23.318951+0000","last_unstale":"2023-10-06T02:45:27.334989+0000","last_undegraded":"2023-10-06T02:45:27.334989+0000","last_fullsized":"2023-10-06T02:45:27.334989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:34.946622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325697+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:45:27.325697+0000","last_peered":"2023-10-06T02:45:27.325697+0000","last_clean":"2023-10-06T02:45:27.325697+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:45:27.325697+0000","last_undegraded":"2023-10-06T02:45:27.325697+0000","last_fullsized":"2023-10-06T02:45:27.325697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:28.296097+0000","last_change":"2023-10-06T02:45:22.799764+0000","last_active":"2023-10-06T02:45:28.296097+0000","last_peered":"2023-10-06T02:45:28.296097+0000","last_clean":"2023-10-06T02:45:28.296097+0000","last_became_active":"2023-10-06T02:45:22.798917+0000","last_became_peered":"2023-10-06T02:45:22.798917+0000","last_unstale":"2023-10-06T02:45:28.296097+0000","last_undegraded":"2023-10-06T02:45:28.296097+0000","last_fullsized":"2023-10-06T02:45:28.296097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353993+0000","last_change":"2023-10-06T02:45:27.350164+0000","last_active":"2023-10-06T02:45:27.353993+0000","last_peered":"2023-10-06T02:45:27.353993+0000","last_clean":"2023-10-06T02:45:27.353993+0000","last_became_active":"2023-10-06T02:45:27.349731+0000","last_became_peered":"2023-10-06T02:45:27.349731+0000","last_unstale":"2023-10-06T02:45:27.353993+0000","last_undegraded":"2023-10-06T02:45:27.353993+0000","last_fullsized":"2023-10-06T02:45:27.353993+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:07.179714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332977+0000","last_change":"2023-10-06T02:45:23.324382+0000","last_active":"2023-10-06T02:45:27.332977+0000","last_peered":"2023-10-06T02:45:27.332977+0000","last_clean":"2023-10-06T02:45:27.332977+0000","last_became_active":"2023-10-06T02:45:23.324017+0000","last_became_peered":"2023-10-06T02:45:23.324017+0000","last_unstale":"2023-10-06T02:45:27.332977+0000","last_undegraded":"2023-10-06T02:45:27.332977+0000","last_fullsized":"2023-10-06T02:45:27.332977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:01.347856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340460+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:45:27.340460+0000","last_peered":"2023-10-06T02:45:27.340460+0000","last_clean":"2023-10-06T02:45:27.340460+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:45:27.340460+0000","last_undegraded":"2023-10-06T02:45:27.340460+0000","last_fullsized":"2023-10-06T02:45:27.340460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333588+0000","last_change":"2023-10-06T02:45:22.807333+0000","last_active":"2023-10-06T02:45:27.333588+0000","last_peered":"2023-10-06T02:45:27.333588+0000","last_clean":"2023-10-06T02:45:27.333588+0000","last_became_active":"2023-10-06T02:45:22.805988+0000","last_became_peered":"2023-10-06T02:45:22.805988+0000","last_unstale":"2023-10-06T02:45:27.333588+0000","last_undegraded":"2023-10-06T02:45:27.333588+0000","last_fullsized":"2023-10-06T02:45:27.333588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:41.870262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353464+0000","last_change":"2023-10-06T02:45:27.347859+0000","last_active":"2023-10-06T02:45:27.353464+0000","last_peered":"2023-10-06T02:45:27.353464+0000","last_clean":"2023-10-06T02:45:27.353464+0000","last_became_active":"2023-10-06T02:45:27.347418+0000","last_became_peered":"2023-10-06T02:45:27.347418+0000","last_unstale":"2023-10-06T02:45:27.353464+0000","last_undegraded":"2023-10-06T02:45:27.353464+0000","last_fullsized":"2023-10-06T02:45:27.353464+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:17.541971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334956+0000","last_change":"2023-10-06T02:45:23.320296+0000","last_active":"2023-10-06T02:45:27.334956+0000","last_peered":"2023-10-06T02:45:27.334956+0000","last_clean":"2023-10-06T02:45:27.334956+0000","last_became_active":"2023-10-06T02:45:23.319983+0000","last_became_peered":"2023-10-06T02:45:23.319983+0000","last_unstale":"2023-10-06T02:45:27.334956+0000","last_undegraded":"2023-10-06T02:45:27.334956+0000","last_fullsized":"2023-10-06T02:45:27.334956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333252+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:45:27.333252+0000","last_peered":"2023-10-06T02:45:27.333252+0000","last_clean":"2023-10-06T02:45:27.333252+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:45:27.333252+0000","last_undegraded":"2023-10-06T02:45:27.333252+0000","last_fullsized":"2023-10-06T02:45:27.333252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336685+0000","last_change":"2023-10-06T02:45:22.804019+0000","last_active":"2023-10-06T02:45:27.336685+0000","last_peered":"2023-10-06T02:45:27.336685+0000","last_clean":"2023-10-06T02:45:27.336685+0000","last_became_active":"2023-10-06T02:45:22.803524+0000","last_became_peered":"2023-10-06T02:45:22.803524+0000","last_unstale":"2023-10-06T02:45:27.336685+0000","last_undegraded":"2023-10-06T02:45:27.336685+0000","last_fullsized":"2023-10-06T02:45:27.336685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353592+0000","last_change":"2023-10-06T02:45:27.344027+0000","last_active":"2023-10-06T02:45:27.353592+0000","last_peered":"2023-10-06T02:45:27.353592+0000","last_clean":"2023-10-06T02:45:27.353592+0000","last_became_active":"2023-10-06T02:45:27.343450+0000","last_became_peered":"2023-10-06T02:45:27.343450+0000","last_unstale":"2023-10-06T02:45:27.353592+0000","last_undegraded":"2023-10-06T02:45:27.353592+0000","last_fullsized":"2023-10-06T02:45:27.353592+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:42.951337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336038+0000","last_change":"2023-10-06T02:45:23.319757+0000","last_active":"2023-10-06T02:45:27.336038+0000","last_peered":"2023-10-06T02:45:27.336038+0000","last_clean":"2023-10-06T02:45:27.336038+0000","last_became_active":"2023-10-06T02:45:23.319356+0000","last_became_peered":"2023-10-06T02:45:23.319356+0000","last_unstale":"2023-10-06T02:45:27.336038+0000","last_undegraded":"2023-10-06T02:45:27.336038+0000","last_fullsized":"2023-10-06T02:45:27.336038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337310+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:45:27.337310+0000","last_peered":"2023-10-06T02:45:27.337310+0000","last_clean":"2023-10-06T02:45:27.337310+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:45:27.337310+0000","last_undegraded":"2023-10-06T02:45:27.337310+0000","last_fullsized":"2023-10-06T02:45:27.337310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341535+0000","last_change":"2023-10-06T02:45:22.808675+0000","last_active":"2023-10-06T02:45:27.341535+0000","last_peered":"2023-10-06T02:45:27.341535+0000","last_clean":"2023-10-06T02:45:27.341535+0000","last_became_active":"2023-10-06T02:45:22.807693+0000","last_became_peered":"2023-10-06T02:45:22.807693+0000","last_unstale":"2023-10-06T02:45:27.341535+0000","last_undegraded":"2023-10-06T02:45:27.341535+0000","last_fullsized":"2023-10-06T02:45:27.341535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356445+0000","last_change":"2023-10-06T02:45:27.352852+0000","last_active":"2023-10-06T02:45:27.356445+0000","last_peered":"2023-10-06T02:45:27.356445+0000","last_clean":"2023-10-06T02:45:27.356445+0000","last_became_active":"2023-10-06T02:45:27.352303+0000","last_became_peered":"2023-10-06T02:45:27.352303+0000","last_unstale":"2023-10-06T02:45:27.356445+0000","last_undegraded":"2023-10-06T02:45:27.356445+0000","last_fullsized":"2023-10-06T02:45:27.356445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:21.854256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336597+0000","last_change":"2023-10-06T02:45:23.317570+0000","last_active":"2023-10-06T02:45:27.336597+0000","last_peered":"2023-10-06T02:45:27.336597+0000","last_clean":"2023-10-06T02:45:27.336597+0000","last_became_active":"2023-10-06T02:45:23.316419+0000","last_became_peered":"2023-10-06T02:45:23.316419+0000","last_unstale":"2023-10-06T02:45:27.336597+0000","last_undegraded":"2023-10-06T02:45:27.336597+0000","last_fullsized":"2023-10-06T02:45:27.336597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332685+0000","last_change":"2023-10-06T02:45:22.799727+0000","last_active":"2023-10-06T02:45:27.332685+0000","last_peered":"2023-10-06T02:45:27.332685+0000","last_clean":"2023-10-06T02:45:27.332685+0000","last_became_active":"2023-10-06T02:45:22.799276+0000","last_became_peered":"2023-10-06T02:45:22.799276+0000","last_unstale":"2023-10-06T02:45:27.332685+0000","last_undegraded":"2023-10-06T02:45:27.332685+0000","last_fullsized":"2023-10-06T02:45:27.332685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340308+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:45:27.340308+0000","last_peered":"2023-10-06T02:45:27.340308+0000","last_clean":"2023-10-06T02:45:27.340308+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:45:27.340308+0000","last_undegraded":"2023-10-06T02:45:27.340308+0000","last_fullsized":"2023-10-06T02:45:27.340308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353669+0000","last_change":"2023-10-06T02:45:27.350898+0000","last_active":"2023-10-06T02:45:27.353669+0000","last_peered":"2023-10-06T02:45:27.353669+0000","last_clean":"2023-10-06T02:45:27.353669+0000","last_became_active":"2023-10-06T02:45:27.350489+0000","last_became_peered":"2023-10-06T02:45:27.350489+0000","last_unstale":"2023-10-06T02:45:27.353669+0000","last_undegraded":"2023-10-06T02:45:27.353669+0000","last_fullsized":"2023-10-06T02:45:27.353669+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340516+0000","last_change":"2023-10-06T02:45:23.320581+0000","last_active":"2023-10-06T02:45:27.340516+0000","last_peered":"2023-10-06T02:45:27.340516+0000","last_clean":"2023-10-06T02:45:27.340516+0000","last_became_active":"2023-10-06T02:45:23.320253+0000","last_became_peered":"2023-10-06T02:45:23.320253+0000","last_unstale":"2023-10-06T02:45:27.340516+0000","last_undegraded":"2023-10-06T02:45:27.340516+0000","last_fullsized":"2023-10-06T02:45:27.340516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:15.466869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334594+0000","last_change":"2023-10-06T02:45:22.803922+0000","last_active":"2023-10-06T02:45:27.334594+0000","last_peered":"2023-10-06T02:45:27.334594+0000","last_clean":"2023-10-06T02:45:27.334594+0000","last_became_active":"2023-10-06T02:45:22.803487+0000","last_became_peered":"2023-10-06T02:45:22.803487+0000","last_unstale":"2023-10-06T02:45:27.334594+0000","last_undegraded":"2023-10-06T02:45:27.334594+0000","last_fullsized":"2023-10-06T02:45:27.334594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319371+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:45:27.319371+0000","last_peered":"2023-10-06T02:45:27.319371+0000","last_clean":"2023-10-06T02:45:27.319371+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:45:27.319371+0000","last_undegraded":"2023-10-06T02:45:27.319371+0000","last_fullsized":"2023-10-06T02:45:27.319371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350330+0000","last_change":"2023-10-06T02:45:27.341548+0000","last_active":"2023-10-06T02:45:27.350330+0000","last_peered":"2023-10-06T02:45:27.350330+0000","last_clean":"2023-10-06T02:45:27.350330+0000","last_became_active":"2023-10-06T02:45:27.341116+0000","last_became_peered":"2023-10-06T02:45:27.341116+0000","last_unstale":"2023-10-06T02:45:27.350330+0000","last_undegraded":"2023-10-06T02:45:27.350330+0000","last_fullsized":"2023-10-06T02:45:27.350330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:12.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334501+0000","last_change":"2023-10-06T02:45:23.313236+0000","last_active":"2023-10-06T02:45:27.334501+0000","last_peered":"2023-10-06T02:45:27.334501+0000","last_clean":"2023-10-06T02:45:27.334501+0000","last_became_active":"2023-10-06T02:45:23.312943+0000","last_became_peered":"2023-10-06T02:45:23.312943+0000","last_unstale":"2023-10-06T02:45:27.334501+0000","last_undegraded":"2023-10-06T02:45:27.334501+0000","last_fullsized":"2023-10-06T02:45:27.334501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333980+0000","last_change":"2023-10-06T02:45:22.801183+0000","last_active":"2023-10-06T02:45:27.333980+0000","last_peered":"2023-10-06T02:45:27.333980+0000","last_clean":"2023-10-06T02:45:27.333980+0000","last_became_active":"2023-10-06T02:45:22.800872+0000","last_became_peered":"2023-10-06T02:45:22.800872+0000","last_unstale":"2023-10-06T02:45:27.333980+0000","last_undegraded":"2023-10-06T02:45:27.333980+0000","last_fullsized":"2023-10-06T02:45:27.333980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331616+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:45:27.331616+0000","last_peered":"2023-10-06T02:45:27.331616+0000","last_clean":"2023-10-06T02:45:27.331616+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:45:27.331616+0000","last_undegraded":"2023-10-06T02:45:27.331616+0000","last_fullsized":"2023-10-06T02:45:27.331616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357435+0000","last_change":"2023-10-06T02:45:27.354883+0000","last_active":"2023-10-06T02:45:27.357435+0000","last_peered":"2023-10-06T02:45:27.357435+0000","last_clean":"2023-10-06T02:45:27.357435+0000","last_became_active":"2023-10-06T02:45:27.353904+0000","last_became_peered":"2023-10-06T02:45:27.353904+0000","last_unstale":"2023-10-06T02:45:27.357435+0000","last_undegraded":"2023-10-06T02:45:27.357435+0000","last_fullsized":"2023-10-06T02:45:27.357435+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:54.845793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325620+0000","last_change":"2023-10-06T02:45:23.311475+0000","last_active":"2023-10-06T02:45:27.325620+0000","last_peered":"2023-10-06T02:45:27.325620+0000","last_clean":"2023-10-06T02:45:27.325620+0000","last_became_active":"2023-10-06T02:45:23.311183+0000","last_became_peered":"2023-10-06T02:45:23.311183+0000","last_unstale":"2023-10-06T02:45:27.325620+0000","last_undegraded":"2023-10-06T02:45:27.325620+0000","last_fullsized":"2023-10-06T02:45:27.325620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:38.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331867+0000","last_change":"2023-10-06T02:45:22.800538+0000","last_active":"2023-10-06T02:45:27.331867+0000","last_peered":"2023-10-06T02:45:27.331867+0000","last_clean":"2023-10-06T02:45:27.331867+0000","last_became_active":"2023-10-06T02:45:22.800251+0000","last_became_peered":"2023-10-06T02:45:22.800251+0000","last_unstale":"2023-10-06T02:45:27.331867+0000","last_undegraded":"2023-10-06T02:45:27.331867+0000","last_fullsized":"2023-10-06T02:45:27.331867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:35.553098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340818+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:45:27.340818+0000","last_peered":"2023-10-06T02:45:27.340818+0000","last_clean":"2023-10-06T02:45:27.340818+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:45:27.340818+0000","last_undegraded":"2023-10-06T02:45:27.340818+0000","last_fullsized":"2023-10-06T02:45:27.340818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331708+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:45:27.331708+0000","last_peered":"2023-10-06T02:45:27.331708+0000","last_clean":"2023-10-06T02:45:27.331708+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:45:27.331708+0000","last_undegraded":"2023-10-06T02:45:27.331708+0000","last_fullsized":"2023-10-06T02:45:27.331708+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351388+0000","last_change":"2023-10-06T02:45:27.349035+0000","last_active":"2023-10-06T02:45:27.351388+0000","last_peered":"2023-10-06T02:45:27.351388+0000","last_clean":"2023-10-06T02:45:27.351388+0000","last_became_active":"2023-10-06T02:45:27.348608+0000","last_became_peered":"2023-10-06T02:45:27.348608+0000","last_unstale":"2023-10-06T02:45:27.351388+0000","last_undegraded":"2023-10-06T02:45:27.351388+0000","last_fullsized":"2023-10-06T02:45:27.351388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:18.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340259+0000","last_change":"2023-10-06T02:45:23.310743+0000","last_active":"2023-10-06T02:45:27.340259+0000","last_peered":"2023-10-06T02:45:27.340259+0000","last_clean":"2023-10-06T02:45:27.340259+0000","last_became_active":"2023-10-06T02:45:23.310401+0000","last_became_peered":"2023-10-06T02:45:23.310401+0000","last_unstale":"2023-10-06T02:45:27.340259+0000","last_undegraded":"2023-10-06T02:45:27.340259+0000","last_fullsized":"2023-10-06T02:45:27.340259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333057+0000","last_change":"2023-10-06T02:45:22.807407+0000","last_active":"2023-10-06T02:45:27.333057+0000","last_peered":"2023-10-06T02:45:27.333057+0000","last_clean":"2023-10-06T02:45:27.333057+0000","last_became_active":"2023-10-06T02:45:22.804974+0000","last_became_peered":"2023-10-06T02:45:22.804974+0000","last_unstale":"2023-10-06T02:45:27.333057+0000","last_undegraded":"2023-10-06T02:45:27.333057+0000","last_fullsized":"2023-10-06T02:45:27.333057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:15.722413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351625+0000","last_change":"2023-10-06T02:45:27.346225+0000","last_active":"2023-10-06T02:45:27.351625+0000","last_peered":"2023-10-06T02:45:27.351625+0000","last_clean":"2023-10-06T02:45:27.351625+0000","last_became_active":"2023-10-06T02:45:27.345701+0000","last_became_peered":"2023-10-06T02:45:27.345701+0000","last_unstale":"2023-10-06T02:45:27.351625+0000","last_undegraded":"2023-10-06T02:45:27.351625+0000","last_fullsized":"2023-10-06T02:45:27.351625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:56.453178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334551+0000","last_change":"2023-10-06T02:45:23.310126+0000","last_active":"2023-10-06T02:45:27.334551+0000","last_peered":"2023-10-06T02:45:27.334551+0000","last_clean":"2023-10-06T02:45:27.334551+0000","last_became_active":"2023-10-06T02:45:23.309805+0000","last_became_peered":"2023-10-06T02:45:23.309805+0000","last_unstale":"2023-10-06T02:45:27.334551+0000","last_undegraded":"2023-10-06T02:45:27.334551+0000","last_fullsized":"2023-10-06T02:45:27.334551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336853+0000","last_change":"2023-10-06T02:45:22.804564+0000","last_active":"2023-10-06T02:45:27.336853+0000","last_peered":"2023-10-06T02:45:27.336853+0000","last_clean":"2023-10-06T02:45:27.336853+0000","last_became_active":"2023-10-06T02:45:22.804286+0000","last_became_peered":"2023-10-06T02:45:22.804286+0000","last_unstale":"2023-10-06T02:45:27.336853+0000","last_undegraded":"2023-10-06T02:45:27.336853+0000","last_fullsized":"2023-10-06T02:45:27.336853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357099+0000","last_change":"2023-10-06T02:45:27.354769+0000","last_active":"2023-10-06T02:45:27.357099+0000","last_peered":"2023-10-06T02:45:27.357099+0000","last_clean":"2023-10-06T02:45:27.357099+0000","last_became_active":"2023-10-06T02:45:27.352768+0000","last_became_peered":"2023-10-06T02:45:27.352768+0000","last_unstale":"2023-10-06T02:45:27.357099+0000","last_undegraded":"2023-10-06T02:45:27.357099+0000","last_fullsized":"2023-10-06T02:45:27.357099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336239+0000","last_change":"2023-10-06T02:45:23.310059+0000","last_active":"2023-10-06T02:45:27.336239+0000","last_peered":"2023-10-06T02:45:27.336239+0000","last_clean":"2023-10-06T02:45:27.336239+0000","last_became_active":"2023-10-06T02:45:23.309667+0000","last_became_peered":"2023-10-06T02:45:23.309667+0000","last_unstale":"2023-10-06T02:45:27.336239+0000","last_undegraded":"2023-10-06T02:45:27.336239+0000","last_fullsized":"2023-10-06T02:45:27.336239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:28.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.326136+0000","last_change":"2023-10-06T02:45:22.795986+0000","last_active":"2023-10-06T02:45:27.326136+0000","last_peered":"2023-10-06T02:45:27.326136+0000","last_clean":"2023-10-06T02:45:27.326136+0000","last_became_active":"2023-10-06T02:45:22.795649+0000","last_became_peered":"2023-10-06T02:45:22.795649+0000","last_unstale":"2023-10-06T02:45:27.326136+0000","last_undegraded":"2023-10-06T02:45:27.326136+0000","last_fullsized":"2023-10-06T02:45:27.326136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:52.473292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360933+0000","last_change":"2023-10-06T02:45:27.347788+0000","last_active":"2023-10-06T02:45:27.360933+0000","last_peered":"2023-10-06T02:45:27.360933+0000","last_clean":"2023-10-06T02:45:27.360933+0000","last_became_active":"2023-10-06T02:45:27.347192+0000","last_became_peered":"2023-10-06T02:45:27.347192+0000","last_unstale":"2023-10-06T02:45:27.360933+0000","last_undegraded":"2023-10-06T02:45:27.360933+0000","last_fullsized":"2023-10-06T02:45:27.360933+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336639+0000","last_change":"2023-10-06T02:45:23.307192+0000","last_active":"2023-10-06T02:45:27.336639+0000","last_peered":"2023-10-06T02:45:27.336639+0000","last_clean":"2023-10-06T02:45:27.336639+0000","last_became_active":"2023-10-06T02:45:23.306642+0000","last_became_peered":"2023-10-06T02:45:23.306642+0000","last_unstale":"2023-10-06T02:45:27.336639+0000","last_undegraded":"2023-10-06T02:45:27.336639+0000","last_fullsized":"2023-10-06T02:45:27.336639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:14.468949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333926+0000","last_change":"2023-10-06T02:45:22.811906+0000","last_active":"2023-10-06T02:45:27.333926+0000","last_peered":"2023-10-06T02:45:27.333926+0000","last_clean":"2023-10-06T02:45:27.333926+0000","last_became_active":"2023-10-06T02:45:22.811602+0000","last_became_peered":"2023-10-06T02:45:22.811602+0000","last_unstale":"2023-10-06T02:45:27.333926+0000","last_undegraded":"2023-10-06T02:45:27.333926+0000","last_fullsized":"2023-10-06T02:45:27.333926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.090764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.349516+0000","last_change":"2023-10-06T02:45:27.344016+0000","last_active":"2023-10-06T02:45:27.349516+0000","last_peered":"2023-10-06T02:45:27.349516+0000","last_clean":"2023-10-06T02:45:27.349516+0000","last_became_active":"2023-10-06T02:45:27.342950+0000","last_became_peered":"2023-10-06T02:45:27.342950+0000","last_unstale":"2023-10-06T02:45:27.349516+0000","last_undegraded":"2023-10-06T02:45:27.349516+0000","last_fullsized":"2023-10-06T02:45:27.349516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:25.726296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334869+0000","last_change":"2023-10-06T02:45:23.321013+0000","last_active":"2023-10-06T02:45:27.334869+0000","last_peered":"2023-10-06T02:45:27.334869+0000","last_clean":"2023-10-06T02:45:27.334869+0000","last_became_active":"2023-10-06T02:45:23.320682+0000","last_became_peered":"2023-10-06T02:45:23.320682+0000","last_unstale":"2023-10-06T02:45:27.334869+0000","last_undegraded":"2023-10-06T02:45:27.334869+0000","last_fullsized":"2023-10-06T02:45:27.334869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341800+0000","last_change":"2023-10-06T02:45:22.806416+0000","last_active":"2023-10-06T02:45:27.341800+0000","last_peered":"2023-10-06T02:45:27.341800+0000","last_clean":"2023-10-06T02:45:27.341800+0000","last_became_active":"2023-10-06T02:45:22.806087+0000","last_became_peered":"2023-10-06T02:45:22.806087+0000","last_unstale":"2023-10-06T02:45:27.341800+0000","last_undegraded":"2023-10-06T02:45:27.341800+0000","last_fullsized":"2023-10-06T02:45:27.341800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357937+0000","last_change":"2023-10-06T02:45:27.350897+0000","last_active":"2023-10-06T02:45:27.357937+0000","last_peered":"2023-10-06T02:45:27.357937+0000","last_clean":"2023-10-06T02:45:27.357937+0000","last_became_active":"2023-10-06T02:45:27.350497+0000","last_became_peered":"2023-10-06T02:45:27.350497+0000","last_unstale":"2023-10-06T02:45:27.357937+0000","last_undegraded":"2023-10-06T02:45:27.357937+0000","last_fullsized":"2023-10-06T02:45:27.357937+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:38.481448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325860+0000","last_change":"2023-10-06T02:45:23.313267+0000","last_active":"2023-10-06T02:45:27.325860+0000","last_peered":"2023-10-06T02:45:27.325860+0000","last_clean":"2023-10-06T02:45:27.325860+0000","last_became_active":"2023-10-06T02:45:23.312886+0000","last_became_peered":"2023-10-06T02:45:23.312886+0000","last_unstale":"2023-10-06T02:45:27.325860+0000","last_undegraded":"2023-10-06T02:45:27.325860+0000","last_fullsized":"2023-10-06T02:45:27.325860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:30.923831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333106+0000","last_change":"2023-10-06T02:45:22.809724+0000","last_active":"2023-10-06T02:45:27.333106+0000","last_peered":"2023-10-06T02:45:27.333106+0000","last_clean":"2023-10-06T02:45:27.333106+0000","last_became_active":"2023-10-06T02:45:22.809451+0000","last_became_peered":"2023-10-06T02:45:22.809451+0000","last_unstale":"2023-10-06T02:45:27.333106+0000","last_undegraded":"2023-10-06T02:45:27.333106+0000","last_fullsized":"2023-10-06T02:45:27.333106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:10.054721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.348579+0000","last_change":"2023-10-06T02:45:27.343402+0000","last_active":"2023-10-06T02:45:27.348579+0000","last_peered":"2023-10-06T02:45:27.348579+0000","last_clean":"2023-10-06T02:45:27.348579+0000","last_became_active":"2023-10-06T02:45:27.342892+0000","last_became_peered":"2023-10-06T02:45:27.342892+0000","last_unstale":"2023-10-06T02:45:27.348579+0000","last_undegraded":"2023-10-06T02:45:27.348579+0000","last_fullsized":"2023-10-06T02:45:27.348579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:05.116878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334800+0000","last_change":"2023-10-06T02:45:23.313977+0000","last_active":"2023-10-06T02:45:27.334800+0000","last_peered":"2023-10-06T02:45:27.334800+0000","last_clean":"2023-10-06T02:45:27.334800+0000","last_became_active":"2023-10-06T02:45:23.313656+0000","last_became_peered":"2023-10-06T02:45:23.313656+0000","last_unstale":"2023-10-06T02:45:27.334800+0000","last_undegraded":"2023-10-06T02:45:27.334800+0000","last_fullsized":"2023-10-06T02:45:27.334800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:52.328524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334307+0000","last_change":"2023-10-06T02:45:22.809096+0000","last_active":"2023-10-06T02:45:27.334307+0000","last_peered":"2023-10-06T02:45:27.334307+0000","last_clean":"2023-10-06T02:45:27.334307+0000","last_became_active":"2023-10-06T02:45:22.807367+0000","last_became_peered":"2023-10-06T02:45:22.807367+0000","last_unstale":"2023-10-06T02:45:27.334307+0000","last_undegraded":"2023-10-06T02:45:27.334307+0000","last_fullsized":"2023-10-06T02:45:27.334307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357950+0000","last_change":"2023-10-06T02:45:27.349954+0000","last_active":"2023-10-06T02:45:27.357950+0000","last_peered":"2023-10-06T02:45:27.357950+0000","last_clean":"2023-10-06T02:45:27.357950+0000","last_became_active":"2023-10-06T02:45:27.344601+0000","last_became_peered":"2023-10-06T02:45:27.344601+0000","last_unstale":"2023-10-06T02:45:27.357950+0000","last_undegraded":"2023-10-06T02:45:27.357950+0000","last_fullsized":"2023-10-06T02:45:27.357950+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:55.863350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340458+0000","last_change":"2023-10-06T02:45:23.325407+0000","last_active":"2023-10-06T02:45:27.340458+0000","last_peered":"2023-10-06T02:45:27.340458+0000","last_clean":"2023-10-06T02:45:27.340458+0000","last_became_active":"2023-10-06T02:45:23.325027+0000","last_became_peered":"2023-10-06T02:45:23.325027+0000","last_unstale":"2023-10-06T02:45:27.340458+0000","last_undegraded":"2023-10-06T02:45:27.340458+0000","last_fullsized":"2023-10-06T02:45:27.340458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334293+0000","last_change":"2023-10-06T02:45:22.807221+0000","last_active":"2023-10-06T02:45:27.334293+0000","last_peered":"2023-10-06T02:45:27.334293+0000","last_clean":"2023-10-06T02:45:27.334293+0000","last_became_active":"2023-10-06T02:45:22.806899+0000","last_became_peered":"2023-10-06T02:45:22.806899+0000","last_unstale":"2023-10-06T02:45:27.334293+0000","last_undegraded":"2023-10-06T02:45:27.334293+0000","last_fullsized":"2023-10-06T02:45:27.334293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356009+0000","last_change":"2023-10-06T02:45:27.352601+0000","last_active":"2023-10-06T02:45:27.356009+0000","last_peered":"2023-10-06T02:45:27.356009+0000","last_clean":"2023-10-06T02:45:27.356009+0000","last_became_active":"2023-10-06T02:45:27.352177+0000","last_became_peered":"2023-10-06T02:45:27.352177+0000","last_unstale":"2023-10-06T02:45:27.356009+0000","last_undegraded":"2023-10-06T02:45:27.356009+0000","last_fullsized":"2023-10-06T02:45:27.356009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:57.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357070+0000","last_change":"2023-10-06T02:45:27.354742+0000","last_active":"2023-10-06T02:45:27.357070+0000","last_peered":"2023-10-06T02:45:27.357070+0000","last_clean":"2023-10-06T02:45:27.357070+0000","last_became_active":"2023-10-06T02:45:27.354154+0000","last_became_peered":"2023-10-06T02:45:27.354154+0000","last_unstale":"2023-10-06T02:45:27.357070+0000","last_undegraded":"2023-10-06T02:45:27.357070+0000","last_fullsized":"2023-10-06T02:45:27.357070+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:03.091381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.327140+0000","last_change":"2023-10-06T02:45:22.802084+0000","last_active":"2023-10-06T02:45:27.327140+0000","last_peered":"2023-10-06T02:45:27.327140+0000","last_clean":"2023-10-06T02:45:27.327140+0000","last_became_active":"2023-10-06T02:45:22.801685+0000","last_became_peered":"2023-10-06T02:45:22.801685+0000","last_unstale":"2023-10-06T02:45:27.327140+0000","last_undegraded":"2023-10-06T02:45:27.327140+0000","last_fullsized":"2023-10-06T02:45:27.327140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:31.821785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333425+0000","last_change":"2023-10-06T02:45:23.323734+0000","last_active":"2023-10-06T02:45:27.333425+0000","last_peered":"2023-10-06T02:45:27.333425+0000","last_clean":"2023-10-06T02:45:27.333425+0000","last_became_active":"2023-10-06T02:45:23.323390+0000","last_became_peered":"2023-10-06T02:45:23.323390+0000","last_unstale":"2023-10-06T02:45:27.333425+0000","last_undegraded":"2023-10-06T02:45:27.333425+0000","last_fullsized":"2023-10-06T02:45:27.333425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:09:04.032805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350027+0000","last_change":"2023-10-06T02:45:27.346307+0000","last_active":"2023-10-06T02:45:27.350027+0000","last_peered":"2023-10-06T02:45:27.350027+0000","last_clean":"2023-10-06T02:45:27.350027+0000","last_became_active":"2023-10-06T02:45:27.345807+0000","last_became_peered":"2023-10-06T02:45:27.345807+0000","last_unstale":"2023-10-06T02:45:27.350027+0000","last_undegraded":"2023-10-06T02:45:27.350027+0000","last_fullsized":"2023-10-06T02:45:27.350027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.633950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332194+0000","last_change":"2023-10-06T02:45:22.809079+0000","last_active":"2023-10-06T02:45:27.332194+0000","last_peered":"2023-10-06T02:45:27.332194+0000","last_clean":"2023-10-06T02:45:27.332194+0000","last_became_active":"2023-10-06T02:45:22.807326+0000","last_became_peered":"2023-10-06T02:45:22.807326+0000","last_unstale":"2023-10-06T02:45:27.332194+0000","last_undegraded":"2023-10-06T02:45:27.332194+0000","last_fullsized":"2023-10-06T02:45:27.332194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:32.124040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334582+0000","last_change":"2023-10-06T02:45:23.313113+0000","last_active":"2023-10-06T02:45:27.334582+0000","last_peered":"2023-10-06T02:45:27.334582+0000","last_clean":"2023-10-06T02:45:27.334582+0000","last_became_active":"2023-10-06T02:45:23.312797+0000","last_became_peered":"2023-10-06T02:45:23.312797+0000","last_unstale":"2023-10-06T02:45:27.334582+0000","last_undegraded":"2023-10-06T02:45:27.334582+0000","last_fullsized":"2023-10-06T02:45:27.334582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334672+0000","last_change":"2023-10-06T02:45:22.804236+0000","last_active":"2023-10-06T02:45:27.334672+0000","last_peered":"2023-10-06T02:45:27.334672+0000","last_clean":"2023-10-06T02:45:27.334672+0000","last_became_active":"2023-10-06T02:45:22.802766+0000","last_became_peered":"2023-10-06T02:45:22.802766+0000","last_unstale":"2023-10-06T02:45:27.334672+0000","last_undegraded":"2023-10-06T02:45:27.334672+0000","last_fullsized":"2023-10-06T02:45:27.334672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341457+0000","last_change":"2023-10-06T02:45:23.324554+0000","last_active":"2023-10-06T02:45:27.341457+0000","last_peered":"2023-10-06T02:45:27.341457+0000","last_clean":"2023-10-06T02:45:27.341457+0000","last_became_active":"2023-10-06T02:45:23.324267+0000","last_became_peered":"2023-10-06T02:45:23.324267+0000","last_unstale":"2023-10-06T02:45:27.341457+0000","last_undegraded":"2023-10-06T02:45:27.341457+0000","last_fullsized":"2023-10-06T02:45:27.341457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354813+0000","last_change":"2023-10-06T02:45:27.346623+0000","last_active":"2023-10-06T02:45:27.354813+0000","last_peered":"2023-10-06T02:45:27.354813+0000","last_clean":"2023-10-06T02:45:27.354813+0000","last_became_active":"2023-10-06T02:45:27.346022+0000","last_became_peered":"2023-10-06T02:45:27.346022+0000","last_unstale":"2023-10-06T02:45:27.354813+0000","last_undegraded":"2023-10-06T02:45:27.354813+0000","last_fullsized":"2023-10-06T02:45:27.354813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:40.724648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332912+0000","last_change":"2023-10-06T02:45:22.805769+0000","last_active":"2023-10-06T02:45:27.332912+0000","last_peered":"2023-10-06T02:45:27.332912+0000","last_clean":"2023-10-06T02:45:27.332912+0000","last_became_active":"2023-10-06T02:45:22.805441+0000","last_became_peered":"2023-10-06T02:45:22.805441+0000","last_unstale":"2023-10-06T02:45:27.332912+0000","last_undegraded":"2023-10-06T02:45:27.332912+0000","last_fullsized":"2023-10-06T02:45:27.332912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:41.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.338579+0000","last_change":"2023-10-06T02:45:23.312634+0000","last_active":"2023-10-06T02:45:27.338579+0000","last_peered":"2023-10-06T02:45:27.338579+0000","last_clean":"2023-10-06T02:45:27.338579+0000","last_became_active":"2023-10-06T02:45:23.312292+0000","last_became_peered":"2023-10-06T02:45:23.312292+0000","last_unstale":"2023-10-06T02:45:27.338579+0000","last_undegraded":"2023-10-06T02:45:27.338579+0000","last_fullsized":"2023-10-06T02:45:27.338579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:05.795770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357104+0000","last_change":"2023-10-06T02:45:27.354353+0000","last_active":"2023-10-06T02:45:27.357104+0000","last_peered":"2023-10-06T02:45:27.357104+0000","last_clean":"2023-10-06T02:45:27.357104+0000","last_became_active":"2023-10-06T02:45:27.353428+0000","last_became_peered":"2023-10-06T02:45:27.353428+0000","last_unstale":"2023-10-06T02:45:27.357104+0000","last_undegraded":"2023-10-06T02:45:27.357104+0000","last_fullsized":"2023-10-06T02:45:27.357104+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.149006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325192+0000","last_change":"2023-10-06T02:45:23.309222+0000","last_active":"2023-10-06T02:45:27.325192+0000","last_peered":"2023-10-06T02:45:27.325192+0000","last_clean":"2023-10-06T02:45:27.325192+0000","last_became_active":"2023-10-06T02:45:23.308920+0000","last_became_peered":"2023-10-06T02:45:23.308920+0000","last_unstale":"2023-10-06T02:45:27.325192+0000","last_undegraded":"2023-10-06T02:45:27.325192+0000","last_fullsized":"2023-10-06T02:45:27.325192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:41.399143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329866+0000","last_change":"2023-10-06T02:45:23.312534+0000","last_active":"2023-10-06T02:45:27.329866+0000","last_peered":"2023-10-06T02:45:27.329866+0000","last_clean":"2023-10-06T02:45:27.329866+0000","last_became_active":"2023-10-06T02:45:23.312215+0000","last_became_peered":"2023-10-06T02:45:23.312215+0000","last_unstale":"2023-10-06T02:45:27.329866+0000","last_undegraded":"2023-10-06T02:45:27.329866+0000","last_fullsized":"2023-10-06T02:45:27.329866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:39.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336549+0000","last_change":"2023-10-06T02:45:23.305501+0000","last_active":"2023-10-06T02:45:27.336549+0000","last_peered":"2023-10-06T02:45:27.336549+0000","last_clean":"2023-10-06T02:45:27.336549+0000","last_became_active":"2023-10-06T02:45:23.305062+0000","last_became_peered":"2023-10-06T02:45:23.305062+0000","last_unstale":"2023-10-06T02:45:27.336549+0000","last_undegraded":"2023-10-06T02:45:27.336549+0000","last_fullsized":"2023-10-06T02:45:27.336549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333770+0000","last_change":"2023-10-06T02:45:23.324370+0000","last_active":"2023-10-06T02:45:27.333770+0000","last_peered":"2023-10-06T02:45:27.333770+0000","last_clean":"2023-10-06T02:45:27.333770+0000","last_became_active":"2023-10-06T02:45:23.324082+0000","last_became_peered":"2023-10-06T02:45:23.324082+0000","last_unstale":"2023-10-06T02:45:27.333770+0000","last_undegraded":"2023-10-06T02:45:27.333770+0000","last_fullsized":"2023-10-06T02:45:27.333770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:35.146269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337261+0000","last_change":"2023-10-06T02:45:23.322276+0000","last_active":"2023-10-06T02:45:27.337261+0000","last_peered":"2023-10-06T02:45:27.337261+0000","last_clean":"2023-10-06T02:45:27.337261+0000","last_became_active":"2023-10-06T02:45:23.321381+0000","last_became_peered":"2023-10-06T02:45:23.321381+0000","last_unstale":"2023-10-06T02:45:27.337261+0000","last_undegraded":"2023-10-06T02:45:27.337261+0000","last_fullsized":"2023-10-06T02:45:27.337261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341408+0000","last_change":"2023-10-06T02:45:23.324552+0000","last_active":"2023-10-06T02:45:27.341408+0000","last_peered":"2023-10-06T02:45:27.341408+0000","last_clean":"2023-10-06T02:45:27.341408+0000","last_became_active":"2023-10-06T02:45:23.324145+0000","last_became_peered":"2023-10-06T02:45:23.324145+0000","last_unstale":"2023-10-06T02:45:27.341408+0000","last_undegraded":"2023-10-06T02:45:27.341408+0000","last_fullsized":"2023-10-06T02:45:27.341408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340251+0000","last_change":"2023-10-06T02:45:23.316105+0000","last_active":"2023-10-06T02:45:27.340251+0000","last_peered":"2023-10-06T02:45:27.340251+0000","last_clean":"2023-10-06T02:45:27.340251+0000","last_became_active":"2023-10-06T02:45:23.315133+0000","last_became_peered":"2023-10-06T02:45:23.315133+0000","last_unstale":"2023-10-06T02:45:27.340251+0000","last_undegraded":"2023-10-06T02:45:27.340251+0000","last_fullsized":"2023-10-06T02:45:27.340251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.339997+0000","last_change":"2023-10-06T02:45:23.315937+0000","last_active":"2023-10-06T02:45:27.339997+0000","last_peered":"2023-10-06T02:45:27.339997+0000","last_clean":"2023-10-06T02:45:27.339997+0000","last_became_active":"2023-10-06T02:45:23.315541+0000","last_became_peered":"2023-10-06T02:45:23.315541+0000","last_unstale":"2023-10-06T02:45:27.339997+0000","last_undegraded":"2023-10-06T02:45:27.339997+0000","last_fullsized":"2023-10-06T02:45:27.339997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331530+0000","last_change":"2023-10-06T02:45:23.326576+0000","last_active":"2023-10-06T02:45:27.331530+0000","last_peered":"2023-10-06T02:45:27.331530+0000","last_clean":"2023-10-06T02:45:27.331530+0000","last_became_active":"2023-10-06T02:45:23.326298+0000","last_became_peered":"2023-10-06T02:45:23.326298+0000","last_unstale":"2023-10-06T02:45:27.331530+0000","last_undegraded":"2023-10-06T02:45:27.331530+0000","last_fullsized":"2023-10-06T02:45:27.331530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329766+0000","last_change":"2023-10-06T02:45:23.315079+0000","last_active":"2023-10-06T02:45:27.329766+0000","last_peered":"2023-10-06T02:45:27.329766+0000","last_clean":"2023-10-06T02:45:27.329766+0000","last_became_active":"2023-10-06T02:45:23.314780+0000","last_became_peered":"2023-10-06T02:45:23.314780+0000","last_unstale":"2023-10-06T02:45:27.329766+0000","last_undegraded":"2023-10-06T02:45:27.329766+0000","last_fullsized":"2023-10-06T02:45:27.329766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:22.869594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:23.314883+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:23.314594+0000","last_became_peered":"2023-10-06T02:45:23.314594+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332695+0000","last_change":"2023-10-06T02:45:23.324062+0000","last_active":"2023-10-06T02:45:27.332695+0000","last_peered":"2023-10-06T02:45:27.332695+0000","last_clean":"2023-10-06T02:45:27.332695+0000","last_became_active":"2023-10-06T02:45:23.323735+0000","last_became_peered":"2023-10-06T02:45:23.323735+0000","last_unstale":"2023-10-06T02:45:27.332695+0000","last_undegraded":"2023-10-06T02:45:27.332695+0000","last_fullsized":"2023-10-06T02:45:27.332695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336547+0000","last_change":"2023-10-06T02:45:23.306965+0000","last_active":"2023-10-06T02:45:27.336547+0000","last_peered":"2023-10-06T02:45:27.336547+0000","last_clean":"2023-10-06T02:45:27.336547+0000","last_became_active":"2023-10-06T02:45:23.306269+0000","last_became_peered":"2023-10-06T02:45:23.306269+0000","last_unstale":"2023-10-06T02:45:27.336547+0000","last_undegraded":"2023-10-06T02:45:27.336547+0000","last_fullsized":"2023-10-06T02:45:27.336547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320250+0000","last_change":"2023-10-06T02:45:23.305622+0000","last_active":"2023-10-06T02:45:27.320250+0000","last_peered":"2023-10-06T02:45:27.320250+0000","last_clean":"2023-10-06T02:45:27.320250+0000","last_became_active":"2023-10-06T02:45:23.305323+0000","last_became_peered":"2023-10-06T02:45:23.305323+0000","last_unstale":"2023-10-06T02:45:27.320250+0000","last_undegraded":"2023-10-06T02:45:27.320250+0000","last_fullsized":"2023-10-06T02:45:27.320250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:24.265950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334435+0000","last_change":"2023-10-06T02:45:23.311009+0000","last_active":"2023-10-06T02:45:27.334435+0000","last_peered":"2023-10-06T02:45:27.334435+0000","last_clean":"2023-10-06T02:45:27.334435+0000","last_became_active":"2023-10-06T02:45:23.310671+0000","last_became_peered":"2023-10-06T02:45:23.310671+0000","last_unstale":"2023-10-06T02:45:27.334435+0000","last_undegraded":"2023-10-06T02:45:27.334435+0000","last_fullsized":"2023-10-06T02:45:27.334435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:06.358484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334894+0000","last_change":"2023-10-06T02:45:23.312474+0000","last_active":"2023-10-06T02:45:27.334894+0000","last_peered":"2023-10-06T02:45:27.334894+0000","last_clean":"2023-10-06T02:45:27.334894+0000","last_became_active":"2023-10-06T02:45:23.312147+0000","last_became_peered":"2023-10-06T02:45:23.312147+0000","last_unstale":"2023-10-06T02:45:27.334894+0000","last_undegraded":"2023-10-06T02:45:27.334894+0000","last_fullsized":"2023-10-06T02:45:27.334894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330934+0000","last_change":"2023-10-06T02:45:23.314526+0000","last_active":"2023-10-06T02:45:27.330934+0000","last_peered":"2023-10-06T02:45:27.330934+0000","last_clean":"2023-10-06T02:45:27.330934+0000","last_became_active":"2023-10-06T02:45:23.314229+0000","last_became_peered":"2023-10-06T02:45:23.314229+0000","last_unstale":"2023-10-06T02:45:27.330934+0000","last_undegraded":"2023-10-06T02:45:27.330934+0000","last_fullsized":"2023-10-06T02:45:27.330934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:08.831291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332089+0000","last_change":"2023-10-06T02:45:23.326220+0000","last_active":"2023-10-06T02:45:27.332089+0000","last_peered":"2023-10-06T02:45:27.332089+0000","last_clean":"2023-10-06T02:45:27.332089+0000","last_became_active":"2023-10-06T02:45:23.325853+0000","last_became_peered":"2023-10-06T02:45:23.325853+0000","last_unstale":"2023-10-06T02:45:27.332089+0000","last_undegraded":"2023-10-06T02:45:27.332089+0000","last_fullsized":"2023-10-06T02:45:27.332089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340313+0000","last_change":"2023-10-06T02:45:23.319007+0000","last_active":"2023-10-06T02:45:27.340313+0000","last_peered":"2023-10-06T02:45:27.340313+0000","last_clean":"2023-10-06T02:45:27.340313+0000","last_became_active":"2023-10-06T02:45:23.318521+0000","last_became_peered":"2023-10-06T02:45:23.318521+0000","last_unstale":"2023-10-06T02:45:27.340313+0000","last_undegraded":"2023-10-06T02:45:27.340313+0000","last_fullsized":"2023-10-06T02:45:27.340313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319562+0000","last_change":"2023-10-06T02:45:23.308224+0000","last_active":"2023-10-06T02:45:27.319562+0000","last_peered":"2023-10-06T02:45:27.319562+0000","last_clean":"2023-10-06T02:45:27.319562+0000","last_became_active":"2023-10-06T02:45:23.307908+0000","last_became_peered":"2023-10-06T02:45:23.307908+0000","last_unstale":"2023-10-06T02:45:27.319562+0000","last_undegraded":"2023-10-06T02:45:27.319562+0000","last_fullsized":"2023-10-06T02:45:27.319562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:26.518844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319631+0000","last_change":"2023-10-06T02:45:23.312138+0000","last_active":"2023-10-06T02:45:27.319631+0000","last_peered":"2023-10-06T02:45:27.319631+0000","last_clean":"2023-10-06T02:45:27.319631+0000","last_became_active":"2023-10-06T02:45:23.311612+0000","last_became_peered":"2023-10-06T02:45:23.311612+0000","last_unstale":"2023-10-06T02:45:27.319631+0000","last_undegraded":"2023-10-06T02:45:27.319631+0000","last_fullsized":"2023-10-06T02:45:27.319631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.807177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336407+0000","last_change":"2023-10-06T02:45:23.314057+0000","last_active":"2023-10-06T02:45:27.336407+0000","last_peered":"2023-10-06T02:45:27.336407+0000","last_clean":"2023-10-06T02:45:27.336407+0000","last_became_active":"2023-10-06T02:45:23.313762+0000","last_became_peered":"2023-10-06T02:45:23.313762+0000","last_unstale":"2023-10-06T02:45:27.336407+0000","last_undegraded":"2023-10-06T02:45:27.336407+0000","last_fullsized":"2023-10-06T02:45:27.336407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:54.392205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341622+0000","last_change":"2023-10-06T02:45:23.318996+0000","last_active":"2023-10-06T02:45:27.341622+0000","last_peered":"2023-10-06T02:45:27.341622+0000","last_clean":"2023-10-06T02:45:27.341622+0000","last_became_active":"2023-10-06T02:45:23.318710+0000","last_became_peered":"2023-10-06T02:45:23.318710+0000","last_unstale":"2023-10-06T02:45:27.341622+0000","last_undegraded":"2023-10-06T02:45:27.341622+0000","last_fullsized":"2023-10-06T02:45:27.341622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:04.789619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334795+0000","last_change":"2023-10-06T02:45:23.312737+0000","last_active":"2023-10-06T02:45:27.334795+0000","last_peered":"2023-10-06T02:45:27.334795+0000","last_clean":"2023-10-06T02:45:27.334795+0000","last_became_active":"2023-10-06T02:45:23.312446+0000","last_became_peered":"2023-10-06T02:45:23.312446+0000","last_unstale":"2023-10-06T02:45:27.334795+0000","last_undegraded":"2023-10-06T02:45:27.334795+0000","last_fullsized":"2023-10-06T02:45:27.334795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340404+0000","last_change":"2023-10-06T02:45:23.318887+0000","last_active":"2023-10-06T02:45:27.340404+0000","last_peered":"2023-10-06T02:45:27.340404+0000","last_clean":"2023-10-06T02:45:27.340404+0000","last_became_active":"2023-10-06T02:45:23.318573+0000","last_became_peered":"2023-10-06T02:45:23.318573+0000","last_unstale":"2023-10-06T02:45:27.340404+0000","last_undegraded":"2023-10-06T02:45:27.340404+0000","last_fullsized":"2023-10-06T02:45:27.340404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331986+0000","last_change":"2023-10-06T02:45:23.326294+0000","last_active":"2023-10-06T02:45:27.331986+0000","last_peered":"2023-10-06T02:45:27.331986+0000","last_clean":"2023-10-06T02:45:27.331986+0000","last_became_active":"2023-10-06T02:45:23.325935+0000","last_became_peered":"2023-10-06T02:45:23.325935+0000","last_unstale":"2023-10-06T02:45:27.331986+0000","last_undegraded":"2023-10-06T02:45:27.331986+0000","last_fullsized":"2023-10-06T02:45:27.331986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336417+0000","last_change":"2023-10-06T02:45:23.316621+0000","last_active":"2023-10-06T02:45:27.336417+0000","last_peered":"2023-10-06T02:45:27.336417+0000","last_clean":"2023-10-06T02:45:27.336417+0000","last_became_active":"2023-10-06T02:45:23.316283+0000","last_became_peered":"2023-10-06T02:45:23.316283+0000","last_unstale":"2023-10-06T02:45:27.336417+0000","last_undegraded":"2023-10-06T02:45:27.336417+0000","last_fullsized":"2023-10-06T02:45:27.336417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:21.187543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335121+0000","last_change":"2023-10-06T02:45:23.318488+0000","last_active":"2023-10-06T02:45:27.335121+0000","last_peered":"2023-10-06T02:45:27.335121+0000","last_clean":"2023-10-06T02:45:27.335121+0000","last_became_active":"2023-10-06T02:45:23.318144+0000","last_became_peered":"2023-10-06T02:45:23.318144+0000","last_unstale":"2023-10-06T02:45:27.335121+0000","last_undegraded":"2023-10-06T02:45:27.335121+0000","last_fullsized":"2023-10-06T02:45:27.335121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325094+0000","last_change":"2023-10-06T02:45:23.316019+0000","last_active":"2023-10-06T02:45:27.325094+0000","last_peered":"2023-10-06T02:45:27.325094+0000","last_clean":"2023-10-06T02:45:27.325094+0000","last_became_active":"2023-10-06T02:45:23.315706+0000","last_became_peered":"2023-10-06T02:45:23.315706+0000","last_unstale":"2023-10-06T02:45:27.325094+0000","last_undegraded":"2023-10-06T02:45:27.325094+0000","last_fullsized":"2023-10-06T02:45:27.325094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.498422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336118+0000","last_change":"2023-10-06T02:45:23.313938+0000","last_active":"2023-10-06T02:45:27.336118+0000","last_peered":"2023-10-06T02:45:27.336118+0000","last_clean":"2023-10-06T02:45:27.336118+0000","last_became_active":"2023-10-06T02:45:23.313340+0000","last_became_peered":"2023-10-06T02:45:23.313340+0000","last_unstale":"2023-10-06T02:45:27.336118+0000","last_undegraded":"2023-10-06T02:45:27.336118+0000","last_fullsized":"2023-10-06T02:45:27.336118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:53.828654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341529+0000","last_change":"2023-10-06T02:45:23.321544+0000","last_active":"2023-10-06T02:45:27.341529+0000","last_peered":"2023-10-06T02:45:27.341529+0000","last_clean":"2023-10-06T02:45:27.341529+0000","last_became_active":"2023-10-06T02:45:23.321216+0000","last_became_peered":"2023-10-06T02:45:23.321216+0000","last_unstale":"2023-10-06T02:45:27.341529+0000","last_undegraded":"2023-10-06T02:45:27.341529+0000","last_fullsized":"2023-10-06T02:45:27.341529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320111+0000","last_change":"2023-10-06T02:45:23.309006+0000","last_active":"2023-10-06T02:45:27.320111+0000","last_peered":"2023-10-06T02:45:27.320111+0000","last_clean":"2023-10-06T02:45:27.320111+0000","last_became_active":"2023-10-06T02:45:23.308617+0000","last_became_peered":"2023-10-06T02:45:23.308617+0000","last_unstale":"2023-10-06T02:45:27.320111+0000","last_undegraded":"2023-10-06T02:45:27.320111+0000","last_fullsized":"2023-10-06T02:45:27.320111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:35.295610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":131211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":315392,"data_stored":127531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":718363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:45:42.770 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:45:42.770 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T02:45:42.770 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T02:45:42.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T02:45:42.771 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T02:45:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.873+0000 7f6538f5c700 1 Processor -- start 2023-10-06T02:45:42.880 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6538f5c700 1 -- start start 2023-10-06T02:45:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6538f5c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6534158d00 0x7f65340c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6538f5c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65340c7520 0x7f653415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6538f5c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6534069f60 con 0x7f653406b000 2023-10-06T02:45:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6538f5c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653406a240 con 0x7f6534158d00 2023-10-06T02:45:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6538f5c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653406a520 con 0x7f65340c7520 2023-10-06T02:45:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f65337fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6534158d00 0x7f65340c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6533fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f653406b000 0x7f6534066fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44668/0 (socket says 172.21.15.33:44668) 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6533fff700 1 -- 172.21.15.33:0/3856711650 learned_addr learned my addr 172.21.15.33:0/3856711650 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f652bfff700 1 --2- 172.21.15.33:0/3856711650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65340c7520 0x7f653415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6532ffd700 1 -- 172.21.15.33:0/3856711650 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377609069 0 0) 0x7f6534069f60 con 0x7f653406b000 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6532ffd700 1 -- 172.21.15.33:0/3856711650 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6520003570 con 0x7f653406b000 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f65337fe700 1 -- 172.21.15.33:0/3856711650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65340c7520 msgr2=0x7f653415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f65337fe700 1 --2- 172.21.15.33:0/3856711650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65340c7520 0x7f653415cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f65337fe700 1 -- 172.21.15.33:0/3856711650 >> v1:172.21.15.33:6789/0 conn(0x7f653406b000 legacy=0x7f6534066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f65337fe700 1 -- 172.21.15.33:0/3856711650 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653406a800 con 0x7f6534158d00 2023-10-06T02:45:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f65337fe700 1 --2- 172.21.15.33:0/3856711650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6534158d00 0x7f65340c6fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65240098e0 tx=0x7f65240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3306ff3154d80d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6532ffd700 1 -- 172.21.15.33:0/3856711650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524011630 con 0x7f6534158d00 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6532ffd700 1 -- 172.21.15.33:0/3856711650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6524011790 con 0x7f6534158d00 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.877+0000 7f6532ffd700 1 -- 172.21.15.33:0/3856711650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524011b30 con 0x7f6534158d00 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 -- 172.21.15.33:0/3856711650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6534158d00 msgr2=0x7f65340c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 --2- 172.21.15.33:0/3856711650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6534158d00 0x7f65340c6fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65240098e0 tx=0x7f65240091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 -- 172.21.15.33:0/3856711650 shutdown_connections 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 --2- 172.21.15.33:0/3856711650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f65340c7520 0x7f653415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 --2- 172.21.15.33:0/3856711650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6534158d00 0x7f65340c6fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 -- 172.21.15.33:0/3856711650 >> 172.21.15.33:0/3856711650 conn(0x7f65340c8ae0 msgr2=0x7f65340c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 -- 172.21.15.33:0/3856711650 shutdown_connections 2023-10-06T02:45:42.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 -- 172.21.15.33:0/3856711650 wait complete. 2023-10-06T02:45:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.881+0000 7f6538f5c700 1 Processor -- start 2023-10-06T02:45:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 -- start start 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653406b000 0x7f6534158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6534158d00 0x7f653414c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 0x7f65340aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c6840 con 0x7f65340c63d0 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c5010 con 0x7f653406b000 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c52f0 con 0x7f6534158d00 2023-10-06T02:45:42.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 0x7f65340aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 0x7f65340aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54580/0 (socket says 172.21.15.33:54580) 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 -- 172.21.15.33:0/1861322103 learned_addr learned my addr 172.21.15.33:0/1861322103 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f652bfff700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6534158d00 0x7f653414c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f65337fe700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653406b000 0x7f6534158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 -- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6534158d00 msgr2=0x7f653414c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6534158d00 0x7f653414c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 -- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653406b000 msgr2=0x7f6534158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653406b000 0x7f6534158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 -- 172.21.15.33:0/1861322103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6524009420 con 0x7f65340c63d0 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6533fff700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 0x7f65340aa3e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f652c001460 tx=0x7f652c0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00a8b0 con 0x7f65340c63d0 2023-10-06T02:45:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c00aa10 con 0x7f65340c63d0 2023-10-06T02:45:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65340c55d0 con 0x7f65340c63d0 2023-10-06T02:45:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00ad90 con 0x7f65340c63d0 2023-10-06T02:45:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.885+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6534159300 con 0x7f65340c63d0 2023-10-06T02:45:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.889+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653415aa30 con 0x7f65340c63d0 2023-10-06T02:45:42.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.889+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f652c007300 con 0x7f65340c63d0 2023-10-06T02:45:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.889+0000 7f652b7fe700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6514060550 0x7f6514062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.889+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f652c07cfc0 con 0x7f65340c63d0 2023-10-06T02:45:42.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.889+0000 7f65337fe700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6514060550 0x7f6514062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:42.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.893+0000 7f65337fe700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6514060550 0x7f6514062a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65240091e0 tx=0x7f6524017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:42.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:42.893+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f652c04d140 con 0x7f65340c63d0 2023-10-06T02:45:43.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.093+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6534071040 con 0x7f65340c63d0 2023-10-06T02:45:43.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.101+0000 7f652b7fe700 1 -- 172.21.15.33:0/1861322103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154136 (secure 0 0 0) 0x7f652c050760 con 0x7f65340c63d0 2023-10-06T02:45:43.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6514060550 msgr2=0x7f6514062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6514060550 0x7f6514062a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65240091e0 tx=0x7f6524017000 comp rx=0 tx=0).stop 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 msgr2=0x7f65340aa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 0x7f65340aa3e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f652c001460 tx=0x7f652c0019e0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 shutdown_connections 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6514060550 0x7f6514062a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6534158d00 0x7f653414c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65340c63d0 0x7f65340aa3e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 --2- 172.21.15.33:0/1861322103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653406b000 0x7f6534158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 >> 172.21.15.33:0/1861322103 conn(0x7f65340c8ae0 msgr2=0x7f65340ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 shutdown_connections 2023-10-06T02:45:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.105+0000 7f6538f5c700 1 -- 172.21.15.33:0/1861322103 wait complete. 2023-10-06T02:45:43.126 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":33311},{"type":"v1","addr":"172.21.15.167:6833","nonce":33311}]},"active_addr":"172.21.15.167:6833/33311","active_change":"2023-10-06T02:45:02.538274+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":819812474}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":533859293}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1567033892}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1409764636}]}]} 2023-10-06T02:45:43.130 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T02:45:43.130 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T02:45:43.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 Processor -- start 2023-10-06T02:45:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 -- start start 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5c0c0acac0 0x7f5c0c0c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c06cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c0c8ff0 con 0x7f5c0c0c8080 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0ad2f0 con 0x7f5c0c0acac0 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.233+0000 7f5c11005700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0cc0e0 con 0x7f5c0c0ace90 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c06cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c0bfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5c0c0acac0 0x7f5c0c0c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c06cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54472/0 (socket says 172.21.15.33:54472) 2023-10-06T02:45:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 -- 172.21.15.33:0/2961952017 learned_addr learned my addr 172.21.15.33:0/2961952017 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c0b7fe700 1 --1- 172.21.15.33:0/2961952017 >> v1:172.21.15.167:6789/0 conn(0x7f5c0c0c8080 0x7f5c0c0ad4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33608/0 (socket says 172.21.15.33:33608) 2023-10-06T02:45:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 -- 172.21.15.33:0/2961952017 >> v2:172.21.15.167:3300/0 conn(0x7f5c0c0acac0 msgr2=0x7f5c0c0c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 --2- 172.21.15.33:0/2961952017 >> v2:172.21.15.167:3300/0 conn(0x7f5c0c0acac0 0x7f5c0c0c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 -- 172.21.15.33:0/2961952017 >> v1:172.21.15.167:6789/0 conn(0x7f5c0c0c8080 legacy=0x7f5c0c0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 -- 172.21.15.33:0/2961952017 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0ce310 con 0x7f5c0c0ace90 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c0bfff700 1 --2- 172.21.15.33:0/2961952017 >> v2:172.21.15.167:3300/0 conn(0x7f5c0c0acac0 0x7f5c0c0c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c10804700 1 --2- 172.21.15.33:0/2961952017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c06cf30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5c040099b0 tx=0x7f5c04009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b527235765dc5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c0affd700 1 -- 172.21.15.33:0/2961952017 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04009ea0 con 0x7f5c0c0ace90 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c0affd700 1 -- 172.21.15.33:0/2961952017 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400b860 con 0x7f5c0c0ace90 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c0affd700 1 -- 172.21.15.33:0/2961952017 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400bc00 con 0x7f5c0c0ace90 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 -- 172.21.15.33:0/2961952017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 msgr2=0x7f5c0c06cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 --2- 172.21.15.33:0/2961952017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c06cf30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5c040099b0 tx=0x7f5c04009250 comp rx=0 tx=0).stop 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 -- 172.21.15.33:0/2961952017 shutdown_connections 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 --2- 172.21.15.33:0/2961952017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c06cf30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 --2- 172.21.15.33:0/2961952017 >> v2:172.21.15.167:3300/0 conn(0x7f5c0c0acac0 0x7f5c0c0c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 -- 172.21.15.33:0/2961952017 >> 172.21.15.33:0/2961952017 conn(0x7f5c0c06c280 msgr2=0x7f5c0c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 -- 172.21.15.33:0/2961952017 shutdown_connections 2023-10-06T02:45:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.237+0000 7f5c11005700 1 -- 172.21.15.33:0/2961952017 wait complete. 2023-10-06T02:45:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 Processor -- start 2023-10-06T02:45:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 -- start start 2023-10-06T02:45:43.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0acac0 0x7f5c0c0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c0ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 0x7f5c0c0c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b07e0 con 0x7f5c0c0c8080 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b1080 con 0x7f5c0c0acac0 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c11005700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c069710 con 0x7f5c0c0ace90 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 0x7f5c0c0c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 0x7f5c0c0c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54582/0 (socket says 172.21.15.33:54582) 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 -- 172.21.15.33:0/1760625191 learned_addr learned my addr 172.21.15.33:0/1760625191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c0bfff700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0acac0 0x7f5c0c0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c0b7fe700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c0ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 -- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 msgr2=0x7f5c0c0ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c0ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 -- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0acac0 msgr2=0x7f5c0c0b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0acac0 0x7f5c0c0b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.241+0000 7f5c10804700 1 -- 172.21.15.33:0/1760625191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc009420 con 0x7f5c0c0c8080 2023-10-06T02:45:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c10804700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 0x7f5c0c0c27e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006fd0 tx=0x7f5c04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04011420 con 0x7f5c0c0c8080 2023-10-06T02:45:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c04011580 con 0x7f5c0c0c8080 2023-10-06T02:45:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c040094f0 con 0x7f5c0c0c8080 2023-10-06T02:45:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04011940 con 0x7f5c0c0c8080 2023-10-06T02:45:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c0b0a30 con 0x7f5c0c0c8080 2023-10-06T02:45:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c071040 con 0x7f5c0c0c8080 2023-10-06T02:45:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5c04011cf0 con 0x7f5c0c0c8080 2023-10-06T02:45:43.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c03fff700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5bec060550 0x7f5bec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.245+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5c0407e660 con 0x7f5c0c0c8080 2023-10-06T02:45:43.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.249+0000 7f5c0bfff700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5bec060550 0x7f5bec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.249+0000 7f5c0bfff700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5bec060550 0x7f5bec062a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009ea0 tx=0x7f5bfc0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.253+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c0404e800 con 0x7f5c0c0c8080 2023-10-06T02:45:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.421+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c0c0b11c0 con 0x7f5c0c0c8080 2023-10-06T02:45:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c03fff700 1 -- 172.21.15.33:0/1760625191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f5c04051e20 con 0x7f5c0c0c8080 2023-10-06T02:45:43.431 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:43.431 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5bec060550 msgr2=0x7f5bec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5bec060550 0x7f5bec062a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009ea0 tx=0x7f5bfc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 msgr2=0x7f5c0c0c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 0x7f5c0c0c27e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006fd0 tx=0x7f5c04017000 comp rx=0 tx=0).stop 2023-10-06T02:45:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 shutdown_connections 2023-10-06T02:45:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5bec060550 0x7f5bec062a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c0c0ace90 0x7f5c0c0ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c0c0c8080 0x7f5c0c0c27e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 --2- 172.21.15.33:0/1760625191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c0c0acac0 0x7f5c0c0b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 >> 172.21.15.33:0/1760625191 conn(0x7f5c0c06c280 msgr2=0x7f5c0c0cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 shutdown_connections 2023-10-06T02:45:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.425+0000 7f5c11005700 1 -- 172.21.15.33:0/1760625191 wait complete. 2023-10-06T02:45:43.445 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T02:45:43.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 Processor -- start 2023-10-06T02:45:43.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 -- start start 2023-10-06T02:45:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d4156b10 0x7f28d4157b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 0x7f28d40cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d415b620 con 0x7f28d40cbe10 2023-10-06T02:45:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d40cc7a0 con 0x7f28d40c9a80 2023-10-06T02:45:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28db41d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41580c0 con 0x7f28d4156b10 2023-10-06T02:45:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28dac1c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 0x7f28d40cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.545+0000 7f28da41b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d4156b10 0x7f28d4157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28d9c1a700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f28d40cbe10 0x7f28d40ca070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33620/0 (socket says 172.21.15.33:33620) 2023-10-06T02:45:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28dac1c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 0x7f28d40cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43626/0 (socket says 172.21.15.33:43626) 2023-10-06T02:45:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28d9c1a700 1 -- 172.21.15.33:0/2587165445 learned_addr learned my addr 172.21.15.33:0/2587165445 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28da41b700 1 -- 172.21.15.33:0/2587165445 >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 msgr2=0x7f28d40cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28da41b700 1 --2- 172.21.15.33:0/2587165445 >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 0x7f28d40cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28da41b700 1 -- 172.21.15.33:0/2587165445 >> v1:172.21.15.167:6789/0 conn(0x7f28d40cbe10 legacy=0x7f28d40ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28d9419700 1 -- 172.21.15.33:0/2587165445 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484497884 0 0) 0x7f28d415b620 con 0x7f28d40cbe10 2023-10-06T02:45:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28da41b700 1 -- 172.21.15.33:0/2587165445 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d415a730 con 0x7f28d4156b10 2023-10-06T02:45:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28dac1c700 1 --2- 172.21.15.33:0/2587165445 >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 0x7f28d40cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28da41b700 1 --2- 172.21.15.33:0/2587165445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d4156b10 0x7f28d4157b80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28c40098d0 tx=0x7f28c4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1c5460095e3552 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28d9419700 1 -- 172.21.15.33:0/2587165445 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4005c10 con 0x7f28d4156b10 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28d9419700 1 -- 172.21.15.33:0/2587165445 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4005d70 con 0x7f28d4156b10 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28d9419700 1 -- 172.21.15.33:0/2587165445 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4006280 con 0x7f28d4156b10 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 -- 172.21.15.33:0/2587165445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d4156b10 msgr2=0x7f28d4157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 --2- 172.21.15.33:0/2587165445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d4156b10 0x7f28d4157b80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28c40098d0 tx=0x7f28c4009120 comp rx=0 tx=0).stop 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 -- 172.21.15.33:0/2587165445 shutdown_connections 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 --2- 172.21.15.33:0/2587165445 >> v2:172.21.15.167:3300/0 conn(0x7f28d40c9a80 0x7f28d40cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 --2- 172.21.15.33:0/2587165445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d4156b10 0x7f28d4157b80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 -- 172.21.15.33:0/2587165445 >> 172.21.15.33:0/2587165445 conn(0x7f28d4086ff0 msgr2=0x7f28d4096cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 -- 172.21.15.33:0/2587165445 shutdown_connections 2023-10-06T02:45:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 -- 172.21.15.33:0/2587165445 wait complete. 2023-10-06T02:45:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 Processor -- start 2023-10-06T02:45:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.549+0000 7f28db41d700 1 -- start start 2023-10-06T02:45:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 0x7f28d40aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d40cbe10 0x7f28d4152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4156b10 0x7f28d4159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4152770 con 0x7f28d40c9a80 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40ab850 con 0x7f28d4156b10 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41599d0 con 0x7f28d40cbe10 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 0x7f28d40aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 0x7f28d40aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54596/0 (socket says 172.21.15.33:54596) 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 -- 172.21.15.33:0/182596881 learned_addr learned my addr 172.21.15.33:0/182596881 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28d9c1a700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d40cbe10 0x7f28d4152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28dac1c700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4156b10 0x7f28d4159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 -- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d40cbe10 msgr2=0x7f28d4152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d40cbe10 0x7f28d4152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 -- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4156b10 msgr2=0x7f28d4159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4156b10 0x7f28d4159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 -- 172.21.15.33:0/182596881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc009a20 con 0x7f28d40c9a80 2023-10-06T02:45:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28da41b700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 0x7f28d40aa5f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009f70 tx=0x7f28c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c401d780 con 0x7f28d40c9a80 2023-10-06T02:45:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c401d8e0 con 0x7f28d40c9a80 2023-10-06T02:45:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c40093c0 con 0x7f28d40c9a80 2023-10-06T02:45:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c401dca0 con 0x7f28d40c9a80 2023-10-06T02:45:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.553+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d40aaf20 con 0x7f28d40c9a80 2023-10-06T02:45:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.557+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f28c401b690 con 0x7f28d40c9a80 2023-10-06T02:45:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.557+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d4071040 con 0x7f28d40c9a80 2023-10-06T02:45:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.557+0000 7f28ca7fc700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f28b8060550 0x7f28b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.557+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f28c407eb30 con 0x7f28d40c9a80 2023-10-06T02:45:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.557+0000 7f28d9c1a700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f28b8060550 0x7f28b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.557+0000 7f28d9c1a700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f28b8060550 0x7f28b8062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28d415c570 tx=0x7f28d0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.561+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f28c404ed90 con 0x7f28d40c9a80 2023-10-06T02:45:43.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.725+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28d406bf70 con 0x7f28d40c9a80 2023-10-06T02:45:43.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.729+0000 7f28ca7fc700 1 -- 172.21.15.33:0/182596881 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f28c40523b0 con 0x7f28d40c9a80 2023-10-06T02:45:43.735 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:43.735 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:43.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f28b8060550 msgr2=0x7f28b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f28b8060550 0x7f28b8062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f28d415c570 tx=0x7f28d0007d30 comp rx=0 tx=0).stop 2023-10-06T02:45:43.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 msgr2=0x7f28d40aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 0x7f28d40aa5f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009f70 tx=0x7f28c4016000 comp rx=0 tx=0).stop 2023-10-06T02:45:43.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 shutdown_connections 2023-10-06T02:45:43.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f28b8060550 0x7f28b8062a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d40cbe10 0x7f28d4152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d4156b10 0x7f28d4159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 --2- 172.21.15.33:0/182596881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28d40c9a80 0x7f28d40aa5f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 >> 172.21.15.33:0/182596881 conn(0x7f28d4086ff0 msgr2=0x7f28d406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 shutdown_connections 2023-10-06T02:45:43.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.733+0000 7f28db41d700 1 -- 172.21.15.33:0/182596881 wait complete. 2023-10-06T02:45:43.750 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T02:45:43.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T02:45:43.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T02:45:43.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T02:45:43.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T02:45:43.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T02:45:43.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T02:45:43.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T02:45:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.889+0000 7f0fe8e3d700 1 Processor -- start 2023-10-06T02:45:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.889+0000 7f0fe8e3d700 1 -- start start 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe8e3d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe8e3d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe40c2820 con 0x7f0fe40c1680 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe8e3d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe40c1c10 con 0x7f0fe414f660 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe8e3d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40c1ef0 con 0x7f0fe414f290 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe2d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0fe40c1680 0x7f0fe40ad550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44670/0 (socket says 172.21.15.33:44670) 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe2d9d700 1 -- 172.21.15.33:0/2632172825 learned_addr learned my addr 172.21.15.33:0/2632172825 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe259c700 1 --2- 172.21.15.33:0/2632172825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe159a700 1 -- 172.21.15.33:0/2632172825 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148512693 0 0) 0x7f0fe40c2820 con 0x7f0fe40c1680 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe159a700 1 -- 172.21.15.33:0/2632172825 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fcc003570 con 0x7f0fe40c1680 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe259c700 1 -- 172.21.15.33:0/2632172825 >> v1:172.21.15.167:6789/0 conn(0x7f0fe414f660 legacy=0x7f0fe40c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe259c700 1 -- 172.21.15.33:0/2632172825 >> v1:172.21.15.33:6789/0 conn(0x7f0fe40c1680 legacy=0x7f0fe40ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe259c700 1 -- 172.21.15.33:0/2632172825 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40c21d0 con 0x7f0fe414f290 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe259c700 1 --2- 172.21.15.33:0/2632172825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40c9f60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00d330 tx=0x7f0fdc007620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef1a690ba1670738 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe159a700 1 -- 172.21.15.33:0/2632172825 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc010070 con 0x7f0fe414f290 2023-10-06T02:45:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe159a700 1 -- 172.21.15.33:0/2632172825 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00dd30 con 0x7f0fe414f290 2023-10-06T02:45:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe159a700 1 -- 172.21.15.33:0/2632172825 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00dea0 con 0x7f0fe414f290 2023-10-06T02:45:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f48e70dd700 1 Processor -- start 2023-10-06T02:45:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/2632172825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 msgr2=0x7f0fe40c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f0fe8e3d700 1 --2- 172.21.15.33:0/2632172825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40c9f60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00d330 tx=0x7f0fdc007620 comp rx=0 tx=0).stop 2023-10-06T02:45:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f551f59e700 1 Processor -- start 2023-10-06T02:45:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f551f59e700 1 -- start start 2023-10-06T02:45:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f552015e970 0x7f55200c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f552005e4b0 0x7f5520069f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5520068010 con 0x7f552015e5a0 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55200c83b0 con 0x7f552005e4b0 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55200ae640 con 0x7f552015e970 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f552005e4b0 0x7f5520069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7fdb6526d700 1 Processor -- start 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7fdb6526d700 1 -- start start 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb600c9550 0x7fdb600683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60169b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdb600864b0 0x7fdb60086910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7fdbe68cb700 1 Processor -- start 2023-10-06T02:45:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7fdbe68cb700 1 -- start start 2023-10-06T02:45:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600ca7e0 con 0x7fdb600864b0 2023-10-06T02:45:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600679c0 con 0x7fdb600c9550 2023-10-06T02:45:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6016a070 con 0x7fdb601696c0 2023-10-06T02:45:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbe68cb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdbe015d340 0x7fdbe00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f552005e4b0 0x7f5520069f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54600/0 (socket says 172.21.15.33:54600) 2023-10-06T02:45:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 -- 172.21.15.33:0/2471909237 learned_addr learned my addr 172.21.15.33:0/2471909237 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 -- 172.21.15.33:0/2471909237 >> v2:172.21.15.167:3300/0 conn(0x7f552015e970 msgr2=0x7f55200c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551dd9b700 1 --2- 172.21.15.33:0/2471909237 >> v2:172.21.15.167:3300/0 conn(0x7f552015e970 0x7f55200c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 --2- 172.21.15.33:0/2471909237 >> v2:172.21.15.167:3300/0 conn(0x7f552015e970 0x7f55200c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbe68cb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe00b2600 con 0x7fdbe00b21e0 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbe68cb700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe0161ba0 con 0x7fdbe00671e0 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbe68cb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe00b1700 con 0x7fdbe015d340 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdbe015d340 0x7fdbe00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdbe015d340 0x7fdbe00675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54614/0 (socket says 172.21.15.33:54614) 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 -- 172.21.15.33:0/467249105 learned_addr learned my addr 172.21.15.33:0/467249105 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.893+0000 7f48e70dd700 1 -- start start 2023-10-06T02:45:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e70dd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48e0159fa0 0x7f48e00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb600c9550 0x7fdb600683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb600c9550 0x7fdb600683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43638/0 (socket says 172.21.15.33:43638) 2023-10-06T02:45:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 -- 172.21.15.33:0/3009933221 learned_addr learned my addr 172.21.15.33:0/3009933221 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e70dd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e00cb5b0 con 0x7f48e0158e70 2023-10-06T02:45:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e70dd700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e0159240 con 0x7f48e00ccc40 2023-10-06T02:45:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e70dd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e015a410 con 0x7f48e0159fa0 2023-10-06T02:45:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5f7fe700 1 --2- 172.21.15.33:0/3009933221 >> v2:172.21.15.33:3300/0 conn(0x7fdb600864b0 0x7fdb60086910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 -- 172.21.15.33:0/2471909237 >> v1:172.21.15.167:6789/0 conn(0x7f552015e5a0 legacy=0x7f55200c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 -- 172.21.15.33:0/2471909237 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200ae8e0 con 0x7f552005e4b0 2023-10-06T02:45:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/2632172825 shutdown_connections 2023-10-06T02:45:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0fe8e3d700 1 --2- 172.21.15.33:0/2632172825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40c9f60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/2632172825 >> 172.21.15.33:0/2632172825 conn(0x7f0fe40c83f0 msgr2=0x7f0fe40c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551ed9d700 1 --2- 172.21.15.33:0/2471909237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552005e4b0 0x7f5520069f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f55200698e0 tx=0x7f5514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99326ece1749da3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e4e79700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f48e0158e70 0x7f48e00cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44694/0 (socket says 172.21.15.33:44694) 2023-10-06T02:45:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e4e79700 1 -- 172.21.15.33:0/3877088734 learned_addr learned my addr 172.21.15.33:0/3877088734 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 -- 172.21.15.33:0/467249105 >> v1:172.21.15.167:6789/0 conn(0x7fdbe00671e0 legacy=0x7fdbe015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 -- 172.21.15.33:0/467249105 >> v1:172.21.15.33:6789/0 conn(0x7fdbe00b21e0 legacy=0x7fdbe0160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/2632172825 shutdown_connections 2023-10-06T02:45:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 -- 172.21.15.33:0/467249105 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00b19e0 con 0x7fdbe015d340 2023-10-06T02:45:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 -- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 msgr2=0x7fdb60169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 --2- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 -- 172.21.15.33:0/3009933221 >> v2:172.21.15.33:3300/0 conn(0x7fdb600864b0 msgr2=0x7fdb60086910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 --2- 172.21.15.33:0/3009933221 >> v2:172.21.15.33:3300/0 conn(0x7fdb600864b0 0x7fdb60086910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 -- 172.21.15.33:0/3009933221 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60086e50 con 0x7fdb600c9550 2023-10-06T02:45:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdf7fe700 1 --2- 172.21.15.33:0/467249105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015d340 0x7fdbe00675b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40099b0 tx=0x7fdbd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed8b69c101ab649 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551d59a700 1 -- 172.21.15.33:0/2471909237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400b820 con 0x7f552005e4b0 2023-10-06T02:45:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551d59a700 1 -- 172.21.15.33:0/2471909237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551400b980 con 0x7f552005e4b0 2023-10-06T02:45:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5effd700 1 --2- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600c9550 0x7fdb600683d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb6006ced0 tx=0x7fdb50009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b371d72323bc56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551d59a700 1 -- 172.21.15.33:0/2471909237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400bd20 con 0x7f552005e4b0 2023-10-06T02:45:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/2632172825 wait complete. 2023-10-06T02:45:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904162152 0 0) 0x7f48e0159240 con 0x7f48e00ccc40 2023-10-06T02:45:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c8003570 con 0x7f48e00ccc40 2023-10-06T02:45:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48dffff700 1 --2- 172.21.15.33:0/3877088734 >> v2:172.21.15.33:3300/0 conn(0x7f48e0159fa0 0x7f48e00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5dffb700 1 -- 172.21.15.33:0/3009933221 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500092e0 con 0x7fdb600c9550 2023-10-06T02:45:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5dffb700 1 -- 172.21.15.33:0/3009933221 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5000ba20 con 0x7fdb600c9550 2023-10-06T02:45:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965216524 0 0) 0x7f48e00cb5b0 con 0x7f48e0158e70 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e0159240 con 0x7f48e0158e70 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 289279796 0 0) 0x7f48c8003570 con 0x7f48e00ccc40 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 >> v2:172.21.15.33:3300/0 conn(0x7f48e0159fa0 msgr2=0x7f48e00cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 --2- 172.21.15.33:0/3877088734 >> v2:172.21.15.33:3300/0 conn(0x7f48e0159fa0 0x7f48e00cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 >> v1:172.21.15.33:6789/0 conn(0x7f48e0158e70 legacy=0x7f48e00cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e00cda80 con 0x7f48e00ccc40 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- 172.21.15.33:0/2471909237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552005e4b0 msgr2=0x7f5520069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 --2- 172.21.15.33:0/2471909237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552005e4b0 0x7f5520069f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f55200698e0 tx=0x7f5514009250 comp rx=0 tx=0).stop 2023-10-06T02:45:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.1 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f48d80017f0 con 0x7f48e00ccc40 2023-10-06T02:45:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 >> v1:172.21.15.167:6789/0 conn(0x7f48e00ccc40 legacy=0x7f48e00ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdeffd700 1 -- 172.21.15.33:0/467249105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400b860 con 0x7fdbe015d340 2023-10-06T02:45:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdeffd700 1 -- 172.21.15.33:0/467249105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd400b9c0 con 0x7fdbe015d340 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48c8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8004810 0x7f48c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c80051c0 0x7f48c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e015a410 con 0x7f48c8003bf0 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c800c1e0 con 0x7f48c8004810 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c800c3a0 con 0x7f48c80051c0 2023-10-06T02:45:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdbdeffd700 1 -- 172.21.15.33:0/467249105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400bd60 con 0x7fdbe015d340 2023-10-06T02:45:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0fe8e3d700 1 Processor -- start 2023-10-06T02:45:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb5dffb700 1 -- 172.21.15.33:0/3009933221 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000bde0 con 0x7fdb600c9550 2023-10-06T02:45:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- 172.21.15.33:0/2471909237 shutdown_connections 2023-10-06T02:45:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 --2- 172.21.15.33:0/2471909237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552005e4b0 0x7f5520069f70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 --2- 172.21.15.33:0/2471909237 >> v2:172.21.15.167:3300/0 conn(0x7f552015e970 0x7f55200c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- 172.21.15.33:0/2471909237 >> 172.21.15.33:0/2471909237 conn(0x7f55200c8ae0 msgr2=0x7f55200c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e4e79700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48c8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7ff385c9d700 1 Processor -- start 2023-10-06T02:45:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f48e567a700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c80051c0 0x7f48c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7ff385c9d700 1 -- start start 2023-10-06T02:45:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 -- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600c9550 msgr2=0x7fdb600683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7fdb6526d700 1 --2- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600c9550 0x7fdb600683d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb6006ced0 tx=0x7fdb50009840 comp rx=0 tx=0).stop 2023-10-06T02:45:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f551f59e700 1 -- 172.21.15.33:0/2471909237 shutdown_connections 2023-10-06T02:45:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 -- start start 2023-10-06T02:45:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 -- 172.21.15.33:0/2471909237 wait complete. 2023-10-06T02:45:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 -- 172.21.15.33:0/3009933221 shutdown_connections 2023-10-06T02:45:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 --2- 172.21.15.33:0/3009933221 >> v2:172.21.15.33:3300/0 conn(0x7fdb600864b0 0x7fdb60086910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 --2- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 --2- 172.21.15.33:0/3009933221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600c9550 0x7fdb600683d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 -- 172.21.15.33:0/3009933221 >> 172.21.15.33:0/3009933221 conn(0x7fdb600c83f0 msgr2=0x7fdb600c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.897+0000 7f0c3359e700 1 Processor -- start 2023-10-06T02:45:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48dffff700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8004810 0x7f48c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3359e700 1 -- start start 2023-10-06T02:45:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e4e79700 1 -- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c80051c0 msgr2=0x7f48c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e4e79700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c80051c0 0x7f48c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e4e79700 1 -- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8004810 msgr2=0x7f48c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e4e79700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8004810 0x7f48c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e567a700 1 -- 172.21.15.33:0/3877088734 reap_dead start 2023-10-06T02:45:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e4e79700 1 -- 172.21.15.33:0/3877088734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8003570 con 0x7f48c8003bf0 2023-10-06T02:45:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e4e79700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48c8003fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48d4001a40 tx=0x7f48d4007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d163c1a4930553e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 -- 172.21.15.33:0/3009933221 shutdown_connections 2023-10-06T02:45:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c1680 0x7f0fe414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe414f660 0x7f0fe40b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40b0bc0 con 0x7f0fe40c1680 2023-10-06T02:45:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40b0290 con 0x7f0fe414f660 2023-10-06T02:45:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40b0570 con 0x7f0fe414f290 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 -- 172.21.15.33:0/467249105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015d340 msgr2=0x7fdbe00675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 --2- 172.21.15.33:0/467249105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015d340 0x7fdbe00675b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40099b0 tx=0x7fdbd4009250 comp rx=0 tx=0).stop 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 -- 172.21.15.33:0/467249105 shutdown_connections 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 --2- 172.21.15.33:0/467249105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015d340 0x7fdbe00675b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 -- 172.21.15.33:0/467249105 >> 172.21.15.33:0/467249105 conn(0x7fdbe00cfd70 msgr2=0x7fdbe00d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 -- 172.21.15.33:0/467249105 shutdown_connections 2023-10-06T02:45:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 -- 172.21.15.33:0/467249105 wait complete. 2023-10-06T02:45:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c1680 0x7f0fe414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c1680 0x7f0fe414e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54652/0 (socket says 172.21.15.33:54652) 2023-10-06T02:45:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 -- 172.21.15.33:0/3140250821 learned_addr learned my addr 172.21.15.33:0/3140250821 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe1d9b700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 -- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 msgr2=0x7f0fe40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0fe414f290 0x7f0fe40adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 -- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe414f660 msgr2=0x7f0fe40b1050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fe414f660 0x7f0fe40b1050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 -- 172.21.15.33:0/3140250821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0078c0 con 0x7f0fe40c1680 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe259c700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c1680 0x7f0fe414e240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fe406ceb0 tx=0x7f0fdc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 Processor -- start 2023-10-06T02:45:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 -- start start 2023-10-06T02:45:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0c340ade00 0x7f0c340c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ca560 0x7f0c340c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 Processor -- start 2023-10-06T02:45:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3359e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3406add0 con 0x7f0c340c5940 2023-10-06T02:45:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c340c80c0 con 0x7f0c340ade00 2023-10-06T02:45:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340c7c40 con 0x7f0c340ca560 2023-10-06T02:45:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c31d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c340c5940 0x7f0c34067840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44706/0 (socket says 172.21.15.33:44706) 2023-10-06T02:45:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c31d9b700 1 -- 172.21.15.33:0/42201859 learned_addr learned my addr 172.21.15.33:0/42201859 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c3259c700 1 --2- 172.21.15.33:0/42201859 >> v2:172.21.15.167:3300/0 conn(0x7f0c340ade00 0x7f0c340c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c32d9d700 1 --2- 172.21.15.33:0/42201859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ca560 0x7f0c340c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc016440 con 0x7f0fe40c1680 2023-10-06T02:45:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40048f0 con 0x7f48c8003bf0 2023-10-06T02:45:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4004a50 con 0x7f48c8003bf0 2023-10-06T02:45:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48df7fe700 1 -- 172.21.15.33:0/3877088734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4004df0 con 0x7f48c8003bf0 2023-10-06T02:45:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 -- 172.21.15.33:0/3009933221 wait complete. 2023-10-06T02:45:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/3140250821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe40b0850 con 0x7f0fe40c1680 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/3140250821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe40ae1c0 con 0x7f0fe40c1680 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552005e4b0 0x7f5520159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552015e5a0 0x7f552015a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552015e970 0x7f552015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552015b680 con 0x7f552015e970 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55200c0030 con 0x7f552015e5a0 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5520161100 con 0x7f552005e4b0 2023-10-06T02:45:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc0165a0 con 0x7f0fe40c1680 2023-10-06T02:45:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc016960 con 0x7f0fe40c1680 2023-10-06T02:45:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552015e5a0 0x7f552015a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552015e970 0x7f552015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552015e5a0 0x7f552015a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43670/0 (socket says 172.21.15.33:43670) 2023-10-06T02:45:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551dd9b700 1 -- 172.21.15.33:0/2874631312 learned_addr learned my addr 172.21.15.33:0/2874631312 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f551ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552015e970 0x7f552015a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54656/0 (socket says 172.21.15.33:54656) 2023-10-06T02:45:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdbe68cb700 1 -- start start 2023-10-06T02:45:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7fdb6526d700 1 Processor -- start 2023-10-06T02:45:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 -- start start 2023-10-06T02:45:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c32d9d700 1 -- 172.21.15.33:0/42201859 >> v2:172.21.15.167:3300/0 conn(0x7f0c340ade00 msgr2=0x7f0c340c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551e59c700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552005e4b0 0x7f5520159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c32d9d700 1 --2- 172.21.15.33:0/42201859 >> v2:172.21.15.167:3300/0 conn(0x7f0c340ade00 0x7f0c340c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c32d9d700 1 -- 172.21.15.33:0/42201859 >> v1:172.21.15.33:6789/0 conn(0x7f0c340c5940 legacy=0x7f0c34067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c32d9d700 1 -- 172.21.15.33:0/42201859 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340ae1d0 con 0x7f0c340ca560 2023-10-06T02:45:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e70dd700 1 -- 172.21.15.33:0/3877088734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 msgr2=0x7f48c8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f48e70dd700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48c8003fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48d4001a40 tx=0x7f48d4007e20 comp rx=0 tx=0).stop 2023-10-06T02:45:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7f0c32d9d700 1 --2- 172.21.15.33:0/42201859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ca560 0x7f0c340c7700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009e60 tx=0x7f0c2c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23acabf9ac3acc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/3140250821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fc8000fd0 con 0x7f0fe40c1680 2023-10-06T02:45:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7ff385c9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3800adf70 0x7ff380154f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7ff385c9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006d810 0x7ff38006a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7ff385c9d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3800c6050 con 0x7ff38006dc60 2023-10-06T02:45:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7ff385c9d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3800673c0 con 0x7ff3800adf70 2023-10-06T02:45:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.901+0000 7ff385c9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38006b430 con 0x7ff38006d810 2023-10-06T02:45:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- 172.21.15.33:0/3877088734 shutdown_connections 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 --2- 172.21.15.33:0/3877088734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48c8003fc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- 172.21.15.33:0/3877088734 >> 172.21.15.33:0/3877088734 conn(0x7f48e0067a80 msgr2=0x7f48e0068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- 172.21.15.33:0/3877088734 shutdown_connections 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- 172.21.15.33:0/3877088734 wait complete. 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551ed9d700 1 -- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552005e4b0 msgr2=0x7f5520159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551ed9d700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f552005e4b0 0x7f5520159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551ed9d700 1 -- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552015e5a0 msgr2=0x7f552015a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551ed9d700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f552015e5a0 0x7f552015a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551ed9d700 1 -- 172.21.15.33:0/2874631312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140094f0 con 0x7f552015e970 2023-10-06T02:45:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551ed9d700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552015e970 0x7f552015a7b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5520069960 tx=0x7f5514016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551401b450 con 0x7f552015e970 2023-10-06T02:45:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5514020070 con 0x7f552015e970 2023-10-06T02:45:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600864b0 0x7fdb60166b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb600c9550 0x7fdb601653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551401b820 con 0x7f552015e970 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552015ade0 con 0x7f552015e970 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552015b3a0 con 0x7f552015e970 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3800adf70 0x7ff380154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff3800adf70 0x7ff380154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54670/0 (socket says 172.21.15.33:54670) 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 -- 172.21.15.33:0/374967385 learned_addr learned my addr 172.21.15.33:0/374967385 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff37ffff700 1 --2- 172.21.15.33:0/374967385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006d810 0x7ff38006a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 -- 172.21.15.33:0/374967385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006d810 msgr2=0x7ff38006a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 --2- 172.21.15.33:0/374967385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006d810 0x7ff38006a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 -- 172.21.15.33:0/374967385 >> v1:172.21.15.167:6789/0 conn(0x7ff38006dc60 legacy=0x7ff38006a7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60173e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c3cc0 con 0x7fdb600c9550 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60168e70 con 0x7fdb600864b0 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb6526d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60167ba0 con 0x7fdb601696c0 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb600c9550 0x7fdb601653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60173e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54558/0 (socket says 172.21.15.33:54558) 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5f7fe700 1 -- 172.21.15.33:0/4091298490 learned_addr learned my addr 172.21.15.33:0/4091298490 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb600c9550 0x7fdb601653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54676/0 (socket says 172.21.15.33:54676) 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 -- 172.21.15.33:0/374967385 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38015bf90 con 0x7ff3800adf70 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 msgr2=0x7fdb60173e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb601696c0 0x7fdb60173e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600864b0 msgr2=0x7fdb60166b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:45:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb600864b0 0x7fdb60166b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/4091298490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50009ae0 con 0x7fdb600c9550 2023-10-06T02:45:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5520071040 con 0x7f552015e970 2023-10-06T02:45:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff384c9b700 1 --2- 172.21.15.33:0/374967385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff380154f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff378009f20 tx=0x7ff378009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e686c36d9a5aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0fdc01d040 con 0x7f0fe40c1680 2023-10-06T02:45:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff37f7fe700 1 -- 172.21.15.33:0/374967385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37800e070 con 0x7ff3800adf70 2023-10-06T02:45:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff37f7fe700 1 -- 172.21.15.33:0/374967385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37800ba20 con 0x7ff3800adf70 2023-10-06T02:45:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7ff37f7fe700 1 -- 172.21.15.33:0/374967385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780092e0 con 0x7ff3800adf70 2023-10-06T02:45:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 Processor -- start 2023-10-06T02:45:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fd27fc700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0fc40606d0 0x7f0fc4062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- start start 2023-10-06T02:45:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbe68cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe00671e0 0x7fdbe01703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbe68cb700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbe00b21e0 0x7fdbe014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbe68cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015c760 0x7fdbe015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbe68cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe015d120 con 0x7fdbe015c760 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbe68cb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe016ff50 con 0x7fdbe00671e0 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbe68cb700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe016f340 con 0x7fdbe00b21e0 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe00671e0 0x7fdbe01703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe00671e0 0x7fdbe01703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43680/0 (socket says 172.21.15.33:43680) 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbdffff700 1 -- 172.21.15.33:0/1564265782 learned_addr learned my addr 172.21.15.33:0/1564265782 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdb5e7fc700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb600c9550 0x7fdb601653a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b630 tx=0x7fdb480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0fdc07e690 con 0x7f0fe40c1680 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fd27fc700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7f0fc406a3a0 0x7f0fc406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fe1d9b700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0fc40606d0 0x7f0fc4062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 --> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fc406cea0 con 0x7f0fc406a3a0 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fdc080c30 con 0x7f0fe40c1680 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fe2d9d700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7f0fc406a3a0 0x7f0fc406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7fdbdf7fe700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbe00b21e0 0x7fdbe014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0c3159a700 1 -- 172.21.15.33:0/42201859 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00e070 con 0x7f0c340ca560 2023-10-06T02:45:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0c3159a700 1 -- 172.21.15.33:0/42201859 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c00bc30 con 0x7f0c340ca560 2023-10-06T02:45:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0c3159a700 1 -- 172.21.15.33:0/42201859 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c012430 con 0x7f0c340ca560 2023-10-06T02:45:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5514022030 con 0x7f552015e970 2023-10-06T02:45:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f0fe2d9d700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7f0fc406a3a0 0x7f0fc406c7d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 -- 172.21.15.33:0/374967385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 msgr2=0x7ff380154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 --2- 172.21.15.33:0/374967385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff380154f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff378009f20 tx=0x7ff378009840 comp rx=0 tx=0).stop 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48e00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48e016cdf0 0x7f48e0149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e00b3ba0 0x7f48e00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e0150d90 con 0x7f48c8003bf0 2023-10-06T02:45:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e0151350 con 0x7f48e00b3ba0 2023-10-06T02:45:43.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.905+0000 7f48e70dd700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e00b4550 con 0x7f48e016cdf0 2023-10-06T02:45:43.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015c760 0x7fdbe015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0fe1d9b700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0fc40606d0 0x7f0fc4062b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0fe415ebf0 tx=0x7f0fd4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48dffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48e016cdf0 0x7f48e0149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48dffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48e016cdf0 0x7f48e0149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54572/0 (socket says 172.21.15.33:54572) 2023-10-06T02:45:43.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48dffff700 1 -- 172.21.15.33:0/4172814822 learned_addr learned my addr 172.21.15.33:0/4172814822 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e00b3ba0 0x7f48e00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e4e79700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48e00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f550e7fc700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55000606a0 0x7f5500062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f551407f350 con 0x7f552015e970 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f551e59c700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55000606a0 0x7f5500062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f550e7fc700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f550006a370 0x7f550006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 -- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbe00b21e0 msgr2=0x7fdbe014b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbe00b21e0 0x7fdbe014b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 -- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe00671e0 msgr2=0x7fdbe01703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbe00671e0 0x7fdbe01703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 -- 172.21.15.33:0/1564265782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd40094f0 con 0x7fdbe015c760 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 -- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48e016cdf0 msgr2=0x7f48e0149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48e016cdf0 0x7f48e0149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 -- 172.21.15.33:0/4172814822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 msgr2=0x7f48e00b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48c8003bf0 0x7f48e00b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 -- 172.21.15.33:0/4172814822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e00b1a50 con 0x7f48e00b3ba0 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800f510 con 0x7fdb600c9550 2023-10-06T02:45:43.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4800f670 con 0x7fdb600c9550 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800fa30 con 0x7fdb600c9550 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb601670d0 con 0x7fdb600c9550 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601675e0 con 0x7fdb600c9550 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 -- 172.21.15.33:0/374967385 shutdown_connections 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 --2- 172.21.15.33:0/374967385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006d810 0x7ff38006a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 --2- 172.21.15.33:0/374967385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff380154f30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 -- 172.21.15.33:0/374967385 >> 172.21.15.33:0/374967385 conn(0x7ff3800cb780 msgr2=0x7ff3800cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f551dd9b700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f550006a370 0x7f550006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe4e68700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015c760 0x7fdbe015cbe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8002230 tx=0x7fdbd8001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f550006ce70 con 0x7f550006a370 2023-10-06T02:45:43.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f551407f680 con 0x7f552015e970 2023-10-06T02:45:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 Processor -- start 2023-10-06T02:45:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 -- start start 2023-10-06T02:45:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e567a700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e00b3ba0 0x7f48e00b4010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48e0068600 tx=0x7f48d8008ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f551dd9b700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f550006a370 0x7f550006c7a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8012630 con 0x7fdbe015c760 2023-10-06T02:45:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb44000fd0 con 0x7fdb600c9550 2023-10-06T02:45:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 -- 172.21.15.33:0/42201859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ca560 msgr2=0x7f0c340c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 --2- 172.21.15.33:0/42201859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ca560 0x7f0c340c7700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009e60 tx=0x7f0c2c009780 comp rx=0 tx=0).stop 2023-10-06T02:45:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd80127d0 con 0x7fdbe015c760 2023-10-06T02:45:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8012b90 con 0x7fdbe015c760 2023-10-06T02:45:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 -- 172.21.15.33:0/42201859 shutdown_connections 2023-10-06T02:45:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 --2- 172.21.15.33:0/42201859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ca560 0x7f0c340c7700 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 --2- 172.21.15.33:0/42201859 >> v2:172.21.15.167:3300/0 conn(0x7f0c340ade00 0x7f0c340c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 -- 172.21.15.33:0/42201859 >> 172.21.15.33:0/42201859 conn(0x7f0c340c8240 msgr2=0x7f0c340c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8003e90 con 0x7f48e00b3ba0 2023-10-06T02:45:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe016f620 con 0x7fdbe015c760 2023-10-06T02:45:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f551e59c700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55000606a0 0x7f5500062b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5510003870 tx=0x7f5510009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe016fbe0 con 0x7fdbe015c760 2023-10-06T02:45:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e00b1d30 con 0x7f48e00b3ba0 2023-10-06T02:45:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e016d240 con 0x7f48e00b3ba0 2023-10-06T02:45:43.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 -- 172.21.15.33:0/374967385 shutdown_connections 2023-10-06T02:45:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== osd.2 v2:172.21.15.33:6808/33422 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f550006ce70 con 0x7f550006a370 2023-10-06T02:45:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 -- 172.21.15.33:0/42201859 shutdown_connections 2023-10-06T02:45:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7ff385c9d700 1 -- 172.21.15.33:0/374967385 wait complete. 2023-10-06T02:45:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d8007bb0 con 0x7f48e00b3ba0 2023-10-06T02:45:43.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d800fb10 con 0x7f48e00b3ba0 2023-10-06T02:45:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdb48024430 con 0x7fdb600c9550 2023-10-06T02:45:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== osd.1 v2:172.21.15.33:6800/33423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0fc406cea0 con 0x7f0fc406a3a0 2023-10-06T02:45:43.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbe00710b0 con 0x7fdbe015c760 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48c0000fd0 con 0x7f48e00b3ba0 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb40060550 0x7fdb40062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdb4807d120 con 0x7fdb600c9550 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdb5effd700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb40060550 0x7fdb40062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdb56ffd700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7fdb4006a220 0x7fdb4006c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb4006cd20 con 0x7fdb4006a220 2023-10-06T02:45:43.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4807d4f0 con 0x7fdb600c9550 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbb4154e30 0x7fcbb406ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcbb40c7220 0x7fcbb40678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40c68f0 con 0x7fcbb4155200 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb40ae210 con 0x7fcbb40c7220 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb9339700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb4067450 con 0x7fcbb4154e30 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7fcbb3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbb4154e30 0x7fcbb406ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcbb4154e30 0x7fcbb406ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43698/0 (socket says 172.21.15.33:43698) 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 -- 172.21.15.33:0/299135492 learned_addr learned my addr 172.21.15.33:0/299135492 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 -- 172.21.15.33:0/299135492 >> v2:172.21.15.33:3300/0 conn(0x7fcbb40c7220 msgr2=0x7fcbb40678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 --2- 172.21.15.33:0/299135492 >> v2:172.21.15.33:3300/0 conn(0x7fcbb40c7220 0x7fcbb40678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 -- 172.21.15.33:0/299135492 >> v1:172.21.15.167:6789/0 conn(0x7fcbb4155200 legacy=0x7fcbb40c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 -- 172.21.15.33:0/299135492 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40c6050 con 0x7fcbb4154e30 2023-10-06T02:45:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdbd8003030 con 0x7fdbe015c760 2023-10-06T02:45:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.909+0000 7f0c3359e700 1 -- 172.21.15.33:0/42201859 wait complete. 2023-10-06T02:45:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 Processor -- start 2023-10-06T02:45:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 Processor -- start 2023-10-06T02:45:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbc7fff700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdbcc060620 0x7fdbcc062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.966 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb3fff700 1 --2- 172.21.15.33:0/299135492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4154e30 0x7fcbb406ad00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40cc750 tx=0x7fcbac00ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4426fbefa66c67c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 -- start start 2023-10-06T02:45:43.967 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdb5effd700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb40060550 0x7fdb40062a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb6006b110 tx=0x7fdb5001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb2ffd700 1 -- 172.21.15.33:0/299135492 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010070 con 0x7fcbb4154e30 2023-10-06T02:45:43.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb2ffd700 1 -- 172.21.15.33:0/299135492 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac00dc80 con 0x7fcbb4154e30 2023-10-06T02:45:43.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb2ffd700 1 -- 172.21.15.33:0/299135492 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac016400 con 0x7fcbb4154e30 2023-10-06T02:45:43.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f48d8015480 con 0x7f48e00b3ba0 2023-10-06T02:45:43.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dcff9700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f48cc060550 0x7f48cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f48d807ca00 con 0x7f48e00b3ba0 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dcff9700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f48cc06a0b0 0x7f48cc06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 --> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48cc06cbb0 con 0x7f48cc06a0b0 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d8081240 con 0x7f48e00b3ba0 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dffff700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f48cc06a0b0 0x7f48cc06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb9339700 1 -- 172.21.15.33:0/299135492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4154e30 msgr2=0x7fcbb406ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fcbb9339700 1 --2- 172.21.15.33:0/299135492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4154e30 0x7fcbb406ad00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40cc750 tx=0x7fcbac00ba50 comp rx=0 tx=0).stop 2023-10-06T02:45:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 -- start start 2023-10-06T02:45:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbdffff700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdbcc060620 0x7fdbcc062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdbd8002280 con 0x7fdbe015c760 2023-10-06T02:45:43.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff38006d810 0x7ff3800c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbc7fff700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7fdbcc06a2f0 0x7fdbcc06c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006dc60 0x7ff38006a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff3800c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbcc06cdf0 con 0x7fdbcc06a2f0 2023-10-06T02:45:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd804cc00 con 0x7fdbe015c760 2023-10-06T02:45:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3800c5770 con 0x7ff3800adf70 2023-10-06T02:45:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3800c5a50 con 0x7ff38006d810 2023-10-06T02:45:43.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff385c9d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3800c30f0 con 0x7ff38006dc60 2023-10-06T02:45:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ade00 0x7f0c34155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c340c5940 0x7f0c340c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff3800c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c340ca560 0x7f0c34175720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff3800c1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54718/0 (socket says 172.21.15.33:54718) 2023-10-06T02:45:43.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff384c9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff38006d810 0x7ff3800c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 -- 172.21.15.33:0/4294857240 learned_addr learned my addr 172.21.15.33:0/4294857240 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341583e0 con 0x7f0c340ca560 2023-10-06T02:45:43.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff37ffff700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006dc60 0x7ff38006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbdf7fe700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7fdbcc06a2f0 0x7fdbcc06c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f48dffff700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f48cc06a0b0 0x7f48cc06c4e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341586a0 con 0x7f0c340c5940 2023-10-06T02:45:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34158980 con 0x7f0c340ade00 2023-10-06T02:45:43.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c340ca560 0x7f0c34175720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c340ca560 0x7f0c34175720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54704/0 (socket says 172.21.15.33:54704) 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 -- 172.21.15.33:0/1098049289 learned_addr learned my addr 172.21.15.33:0/1098049289 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c3259c700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ade00 0x7f0c34155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 -- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006dc60 msgr2=0x7ff38006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff38006dc60 0x7ff38006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 -- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff38006d810 msgr2=0x7ff3800c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff38006d810 0x7ff3800c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 -- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ade00 msgr2=0x7f0c34155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c340ade00 0x7f0c34155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 -- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c340c5940 msgr2=0x7f0c340c50f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:45:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c340c5940 0x7f0c340c50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 -- 172.21.15.33:0/1098049289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c009a20 con 0x7f0c340ca560 2023-10-06T02:45:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 -- 172.21.15.33:0/4294857240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378009ae0 con 0x7ff3800adf70 2023-10-06T02:45:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7fdbdf7fe700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7fdbcc06a2f0 0x7fdbcc06c720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7f0c32d9d700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c340ca560 0x7f0c34175720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c013000 tx=0x7f0c2c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.913+0000 7ff38549c700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff3800c1330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff37000b5e0 tx=0x7ff3700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- 172.21.15.33:0/299135492 shutdown_connections 2023-10-06T02:45:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 --2- 172.21.15.33:0/299135492 >> v2:172.21.15.33:3300/0 conn(0x7fcbb40c7220 0x7fcbb40678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 --2- 172.21.15.33:0/299135492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4154e30 0x7fcbb406ad00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- 172.21.15.33:0/299135492 >> 172.21.15.33:0/299135492 conn(0x7fcbb40cb780 msgr2=0x7fcbb40cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fdb5f7fe700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7fdb4006a220 0x7fdb4006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fdbdffff700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdbcc060620 0x7fdbcc062ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00078a0 tx=0x7fdbd0007830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- 172.21.15.33:0/299135492 shutdown_connections 2023-10-06T02:45:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- 172.21.15.33:0/299135492 wait complete. 2023-10-06T02:45:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fdb5f7fe700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7fdb4006a220 0x7fdb4006c650 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== osd.5 v2:172.21.15.167:6808/33659 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f48cc06cbb0 con 0x7f48cc06a0b0 2023-10-06T02:45:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== osd.3 v2:172.21.15.33:6817/33424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdbcc06cdf0 con 0x7fdbcc06a2f0 2023-10-06T02:45:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c012e50 con 0x7f0c340ca560 2023-10-06T02:45:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c021070 con 0x7f0c340ca560 2023-10-06T02:45:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c34158c60 con 0x7f0c340ca560 2023-10-06T02:45:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c026680 con 0x7f0c340ca560 2023-10-06T02:45:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 Processor -- start 2023-10-06T02:45:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000f510 con 0x7ff3800adf70 2023-10-06T02:45:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37000f6b0 con 0x7ff3800adf70 2023-10-06T02:45:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000fa70 con 0x7ff3800adf70 2023-10-06T02:45:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- start start 2023-10-06T02:45:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== osd.6 v2:172.21.15.167:6816/33656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdb4006cd20 con 0x7fdb4006a220 2023-10-06T02:45:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3800c33d0 con 0x7ff3800adf70 2023-10-06T02:45:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff380144fd0 con 0x7ff3800adf70 2023-10-06T02:45:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c340c5630 con 0x7f0c340ca560 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb40c7220 0x7fcbb4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbb4154e30 0x7fcbb40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4155200 0x7fcbb40b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40b10e0 con 0x7fcbb40c7220 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4169a60 con 0x7fcbb4155200 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb9339700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4169d40 con 0x7fcbb4154e30 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbb4154e30 0x7fcbb40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbb4154e30 0x7fcbb40b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54608/0 (socket says 172.21.15.33:54608) 2023-10-06T02:45:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 -- 172.21.15.33:0/2269942753 learned_addr learned my addr 172.21.15.33:0/2269942753 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff37001a400 con 0x7ff3800adf70 2023-10-06T02:45:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7ff37cff9700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff36c060620 0x7ff36c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff37007c500 con 0x7ff3800adf70 2023-10-06T02:45:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7ff384c9b700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff36c060620 0x7ff36c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 -- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4155200 msgr2=0x7fcbb40b0ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:45:43.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb4155200 0x7fcbb40b0ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 -- 172.21.15.33:0/2269942753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb40c7220 msgr2=0x7fcbb4159c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbb40c7220 0x7fcbb4159c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.917+0000 7fcbb37fe700 1 -- 172.21.15.33:0/2269942753 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac00bcf0 con 0x7fcbb4154e30 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcbb37fe700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbb4154e30 0x7fcbb40b0660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcba400c4e0 tx=0x7fcba400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011590 con 0x7fcbb4154e30 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4011710 con 0x7fcbb4154e30 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011ad0 con 0x7fcbb4154e30 2023-10-06T02:45:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7ff384c9b700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff36c060620 0x7ff36c062ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff378009880 tx=0x7ff378017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0c2c023030 con 0x7f0c340ca560 2023-10-06T02:45:43.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb416a020 con 0x7fcbb4154e30 2023-10-06T02:45:43.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb4169770 con 0x7fcbb4154e30 2023-10-06T02:45:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7f0c2a7fc700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0c14060550 0x7f0c14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0c2c07f020 con 0x7f0c340ca560 2023-10-06T02:45:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7f0c3259c700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0c14060550 0x7f0c14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcbb4071040 con 0x7fcbb4154e30 2023-10-06T02:45:43.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.921+0000 7f0c3259c700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0c14060550 0x7f0c14062a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c24006fd0 tx=0x7f0c24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7ff385c9d700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7ff3600015f0 0x7ff360003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcba401b3b0 con 0x7fcbb4154e30 2023-10-06T02:45:43.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff360006b90 con 0x7ff3600015f0 2023-10-06T02:45:43.990 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f48e4e79700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f48cc060550 0x7f48cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f0c13fff700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f0c34176170 0x7f0c34062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7ff37ffff700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7ff3600015f0 0x7ff360003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.991 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f0c13fff700 1 -- 172.21.15.33:0/1098049289 --> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c3406a1c0 con 0x7f0c34176170 2023-10-06T02:45:43.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f0c31d9b700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f0c34176170 0x7f0c34062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcb9bfff700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcb9c068f40 0x7fcb9c06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7ff37ffff700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7ff3600015f0 0x7ff360003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fcba407c860 con 0x7fcbb4154e30 2023-10-06T02:45:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcb9bfff700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7fcb9c072c10 0x7fcb9c075040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 --> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb9c075710 con 0x7fcb9c072c10 2023-10-06T02:45:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba407cb50 con 0x7fcbb4154e30 2023-10-06T02:45:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcbb8b38700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7fcb9c072c10 0x7fcb9c075040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f0c31d9b700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f0c34176170 0x7f0c34062ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f48e4e79700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f48cc060550 0x7f48cc062a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48d4007e60 tx=0x7f48d4015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== osd.0 v2:172.21.15.33:6816/33421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff360006b90 con 0x7ff3600015f0 2023-10-06T02:45:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcbb3fff700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcb9c068f40 0x7fcb9c06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcbb8b38700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7fcb9c072c10 0x7fcb9c075040 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7fcbb3fff700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcb9c068f40 0x7fcb9c06b3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40cc750 tx=0x7fcbac01c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:43.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.925+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== osd.4 v2:172.21.15.167:6800/33657 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0c3406a1c0 con 0x7f0c34176170 2023-10-06T02:45:43.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.933+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== osd.7 v2:172.21.15.167:6824/33658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fcb9c075710 con 0x7fcb9c072c10 2023-10-06T02:45:43.995 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fe8e3d700 1 -- 172.21.15.33:0/3140250821 --> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fc8002d30 con 0x7f0fc406a3a0 2023-10-06T02:45:43.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 --> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48c0002d30 con 0x7f48cc06a0b0 2023-10-06T02:45:43.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55200c9900 con 0x7f550006a370 2023-10-06T02:45:43.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fd27fc700 1 -- 172.21.15.33:0/3140250821 <== osd.1 v2:172.21.15.33:6800/33423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0fc8002d30 con 0x7f0fc406a3a0 2023-10-06T02:45:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48dcff9700 1 -- 172.21.15.33:0/4172814822 <== osd.5 v2:172.21.15.167:6808/33659 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f48c0002d30 con 0x7f48cc06a0b0 2023-10-06T02:45:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f550e7fc700 1 -- 172.21.15.33:0/2874631312 <== osd.2 v2:172.21.15.33:6808/33422 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f55200c9900 con 0x7f550006a370 2023-10-06T02:45:43.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7f0fc406a3a0 msgr2=0x7f0fc406c7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7f0fc406a3a0 0x7f0fc406c7d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0fc40606d0 msgr2=0x7f0fc4062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f550006a370 msgr2=0x7f550006c7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f550006a370 0x7f550006c7a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f48cc06a0b0 msgr2=0x7f48cc06c4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f48cc06a0b0 0x7f48cc06c4e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55000606a0 msgr2=0x7f5500062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55000606a0 0x7f5500062b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5510003870 tx=0x7f5510009040 comp rx=0 tx=0).stop 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552015e970 msgr2=0x7f552015a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 --2- 172.21.15.33:0/2874631312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f552015e970 0x7f552015a7b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5520069960 tx=0x7f5514016000 comp rx=0 tx=0).stop 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f48cc060550 msgr2=0x7f48cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f48cc060550 0x7f48cc062a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48d4007e60 tx=0x7f48d4015000 comp rx=0 tx=0).stop 2023-10-06T02:45:43.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e00b3ba0 msgr2=0x7f48e00b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 --2- 172.21.15.33:0/4172814822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48e00b3ba0 0x7f48e00b4010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48e0068600 tx=0x7f48d8008ea0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0fc40606d0 0x7f0fc4062b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0fe415ebf0 tx=0x7f0fd4009040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e567a700 1 -- 172.21.15.33:0/4172814822 reap_dead start 2023-10-06T02:45:44.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c1680 msgr2=0x7f0fe414e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 --2- 172.21.15.33:0/3140250821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe40c1680 0x7f0fe414e240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fe406ceb0 tx=0x7f0fdc01b000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fe2d9d700 1 -- 172.21.15.33:0/3140250821 reap_dead start 2023-10-06T02:45:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551ed9d700 1 -- 172.21.15.33:0/2874631312 reap_dead start 2023-10-06T02:45:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 shutdown_connections 2023-10-06T02:45:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 >> 172.21.15.33:0/2874631312 conn(0x7f55200c8ae0 msgr2=0x7f552006dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 shutdown_connections 2023-10-06T02:45:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 >> 172.21.15.33:0/4172814822 conn(0x7f48e0067a80 msgr2=0x7f48e0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 shutdown_connections 2023-10-06T02:45:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 >> 172.21.15.33:0/3140250821 conn(0x7f0fe40c83f0 msgr2=0x7f0fe4157700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 shutdown_connections 2023-10-06T02:45:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f48e70dd700 1 -- 172.21.15.33:0/4172814822 wait complete. 2023-10-06T02:45:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 shutdown_connections 2023-10-06T02:45:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 shutdown_connections 2023-10-06T02:45:44.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f551f59e700 1 -- 172.21.15.33:0/2874631312 wait complete. 2023-10-06T02:45:44.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.941+0000 7f0fc3fff700 1 -- 172.21.15.33:0/3140250821 wait complete. 2023-10-06T02:45:44.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.945+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb44002c90 con 0x7fdb4006a220 2023-10-06T02:45:44.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.945+0000 7fdb56ffd700 1 -- 172.21.15.33:0/4091298490 <== osd.6 v2:172.21.15.167:6816/33656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdb44002c90 con 0x7fdb4006a220 2023-10-06T02:45:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbe00aef30 con 0x7fdbcc06a2f0 2023-10-06T02:45:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7fdb4006a220 msgr2=0x7fdb4006c650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7fdb4006a220 0x7fdb4006c650 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb40060550 msgr2=0x7fdb40062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb40060550 0x7fdb40062a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb6006b110 tx=0x7fdb5001a000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb600c9550 msgr2=0x7fdb601653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 --2- 172.21.15.33:0/4091298490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb600c9550 0x7fdb601653a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b630 tx=0x7fdb480091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb5f7fe700 1 -- 172.21.15.33:0/4091298490 reap_dead start 2023-10-06T02:45:44.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 shutdown_connections 2023-10-06T02:45:44.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 >> 172.21.15.33:0/4091298490 conn(0x7fdb600c83f0 msgr2=0x7fdb600add50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbc7fff700 1 -- 172.21.15.33:0/1564265782 <== osd.3 v2:172.21.15.33:6817/33424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdbe00aef30 con 0x7fdbcc06a2f0 2023-10-06T02:45:44.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 shutdown_connections 2023-10-06T02:45:44.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdb6526d700 1 -- 172.21.15.33:0/4091298490 wait complete. 2023-10-06T02:45:44.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7fdbcc06a2f0 msgr2=0x7fdbcc06c720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7fdbcc06a2f0 0x7fdbcc06c720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdbcc060620 msgr2=0x7fdbcc062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdbcc060620 0x7fdbcc062ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00078a0 tx=0x7fdbd0007830 comp rx=0 tx=0).stop 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015c760 msgr2=0x7fdbe015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 --2- 172.21.15.33:0/1564265782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe015c760 0x7fdbe015cbe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8002230 tx=0x7fdbd8001660 comp rx=0 tx=0).stop 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe4e68700 1 -- 172.21.15.33:0/1564265782 reap_dead start 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 shutdown_connections 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 >> 172.21.15.33:0/1564265782 conn(0x7fdbe00cfd70 msgr2=0x7fdbe00b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 shutdown_connections 2023-10-06T02:45:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.949+0000 7fdbe68cb700 1 -- 172.21.15.33:0/1564265782 wait complete. 2023-10-06T02:45:44.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff360005c80 con 0x7ff3600015f0 2023-10-06T02:45:44.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7f0c13fff700 1 -- 172.21.15.33:0/1098049289 --> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c3405e6b0 con 0x7f0c34176170 2023-10-06T02:45:44.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff37cff9700 1 -- 172.21.15.33:0/4294857240 <== osd.0 v2:172.21.15.33:6816/33421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff360005c80 con 0x7ff3600015f0 2023-10-06T02:45:44.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7f0c2a7fc700 1 -- 172.21.15.33:0/1098049289 <== osd.4 v2:172.21.15.167:6800/33657 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c3405e6b0 con 0x7f0c34176170 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7ff3600015f0 msgr2=0x7ff360003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7ff3600015f0 0x7ff360003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff36c060620 msgr2=0x7ff36c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff36c060620 0x7ff36c062ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff378009880 tx=0x7ff378017000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 msgr2=0x7ff3800c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff385c9d700 1 --2- 172.21.15.33:0/4294857240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3800adf70 0x7ff3800c1330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff37000b5e0 tx=0x7ff3700090f0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.957+0000 7ff38549c700 1 -- 172.21.15.33:0/4294857240 reap_dead start 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 shutdown_connections 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 >> 172.21.15.33:0/4294857240 conn(0x7ff3800cb780 msgr2=0x7ff3800b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 shutdown_connections 2023-10-06T02:45:44.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7ff385c9d700 1 -- 172.21.15.33:0/4294857240 wait complete. 2023-10-06T02:45:44.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 --> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcbb4169e80 con 0x7fcb9c072c10 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f0c34176170 msgr2=0x7f0c34062ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f0c34176170 0x7f0c34062ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0c14060550 msgr2=0x7f0c14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0c14060550 0x7f0c14062a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c24006fd0 tx=0x7f0c24008040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c340ca560 msgr2=0x7f0c34175720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 --2- 172.21.15.33:0/1098049289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c340ca560 0x7f0c34175720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c013000 tx=0x7f0c2c019000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c32d9d700 1 -- 172.21.15.33:0/1098049289 reap_dead start 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 shutdown_connections 2023-10-06T02:45:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 >> 172.21.15.33:0/1098049289 conn(0x7f0c340c8240 msgr2=0x7f0c34153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.014 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:44.014 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:44.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T02:45:44.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T02:45:44.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 shutdown_connections 2023-10-06T02:45:44.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcb9bfff700 1 -- 172.21.15.33:0/2269942753 <== osd.7 v2:172.21.15.167:6824/33658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcbb4169e80 con 0x7fcb9c072c10 2023-10-06T02:45:44.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7f0c3359e700 1 -- 172.21.15.33:0/1098049289 wait complete. 2023-10-06T02:45:44.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7fcb9c072c10 msgr2=0x7fcb9c075040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7fcb9c072c10 0x7fcb9c075040 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcb9c068f40 msgr2=0x7fcb9c06b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fcb9c068f40 0x7fcb9c06b3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40cc750 tx=0x7fcbac01c000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbb4154e30 msgr2=0x7fcbb40b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 --2- 172.21.15.33:0/2269942753 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcbb4154e30 0x7fcbb40b0660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcba400c4e0 tx=0x7fcba400c8a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb8b38700 1 -- 172.21.15.33:0/2269942753 reap_dead start 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 shutdown_connections 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 >> 172.21.15.33:0/2269942753 conn(0x7fcbb40cb780 msgr2=0x7fcbb414dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 shutdown_connections 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:43.961+0000 7fcbb9339700 1 -- 172.21.15.33:0/2269942753 wait complete. 2023-10-06T02:45:44.018 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T02:45:44.020 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:44.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T02:45:44.021 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:44.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T02:45:44.022 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:44.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T02:45:44.023 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:44.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T02:45:44.024 INFO:teuthology.orchestra.run.smithi033.stdout:60129542152 2023-10-06T02:45:44.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T02:45:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 Processor -- start 2023-10-06T02:45:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 -- start start 2023-10-06T02:45:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7fd6f796a700 1 Processor -- start 2023-10-06T02:45:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7fd6f796a700 1 -- start start 2023-10-06T02:45:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ae806a520 0x7f0ae8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ae8154ff0 0x7f0ae80694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae80c79c0 con 0x7f0ae806e4f0 2023-10-06T02:45:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae806a970 con 0x7f0ae8154ff0 2023-10-06T02:45:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aee115700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae80c7e50 con 0x7f0ae806a520 2023-10-06T02:45:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f601e61c700 1 Processor -- start 2023-10-06T02:45:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f601e61c700 1 -- start start 2023-10-06T02:45:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f5af6a61700 1 Processor -- start 2023-10-06T02:45:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aec912700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0ae806a520 0x7f0ae8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aed113700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0ae806e4f0 0x7f0ae80c65b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33680/0 (socket says 172.21.15.33:33680) 2023-10-06T02:45:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aed113700 1 -- 172.21.15.33:0/1526081324 learned_addr learned my addr 172.21.15.33:0/1526081324 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7fd6f796a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6f0160fd0 0x7fd6f00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7fd6f796a700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f0067050 con 0x7fd6f01613a0 2023-10-06T02:45:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7fd6f796a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f0067330 con 0x7fd6f0161820 2023-10-06T02:45:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7fd6f796a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f0161c80 con 0x7fd6f0160fd0 2023-10-06T02:45:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f0aed914700 1 --2- 172.21.15.33:0/1526081324 >> v2:172.21.15.33:3300/0 conn(0x7f0ae8154ff0 0x7f0ae80694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f4f05700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd6f01613a0 0x7fd6f00680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44708/0 (socket says 172.21.15.33:44708) 2023-10-06T02:45:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f4f05700 1 -- 172.21.15.33:0/3844551822 learned_addr learned my addr 172.21.15.33:0/3844551822 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f5706700 1 --2- 172.21.15.33:0/3844551822 >> v2:172.21.15.33:3300/0 conn(0x7fd6f0160fd0 0x7fd6f00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568110509 0 0) 0x7fd6f0067330 con 0x7fd6f0161820 2023-10-06T02:45:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aec912700 1 -- 172.21.15.33:0/1526081324 >> v2:172.21.15.33:3300/0 conn(0x7f0ae8154ff0 msgr2=0x7f0ae80694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aec912700 1 --2- 172.21.15.33:0/1526081324 >> v2:172.21.15.33:3300/0 conn(0x7f0ae8154ff0 0x7f0ae80694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aec912700 1 -- 172.21.15.33:0/1526081324 >> v1:172.21.15.167:6789/0 conn(0x7f0ae806e4f0 legacy=0x7f0ae80c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aec912700 1 -- 172.21.15.33:0/1526081324 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80c6db0 con 0x7f0ae806a520 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.205+0000 7f5af6a61700 1 -- start start 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5af006d6e0 0x7f5af006dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 0x7f5af0067970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af0069d10 con 0x7f5af00b12a0 2023-10-06T02:45:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af00c66c0 con 0x7f5af006d6e0 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af00ae780 con 0x7f5af00bec70 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601e61c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60180c2510 0x7f60180ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601e61c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c0eb0 0x7f60181587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601e61c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60180c2080 con 0x7f60180c9e60 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601e61c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60180c1470 con 0x7f60180c2510 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601e61c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c1750 con 0x7f60180c0eb0 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f6017fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f60180c9e60 0x7f6018068340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44718/0 (socket says 172.21.15.33:44718) 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f6017fff700 1 -- 172.21.15.33:0/690785463 learned_addr learned my addr 172.21.15.33:0/690785463 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601cbb9700 1 --2- 172.21.15.33:0/690785463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c0eb0 0x7f60181587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f60177fe700 1 --2- 172.21.15.33:0/690785463 >> v2:172.21.15.167:3300/0 conn(0x7f60180c2510 0x7f60180ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 0x7f5af0067970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6dc003570 con 0x7fd6f0161820 2023-10-06T02:45:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167304353 0 0) 0x7fd6f0067050 con 0x7fd6f01613a0 2023-10-06T02:45:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f0067330 con 0x7fd6f01613a0 2023-10-06T02:45:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aec912700 1 --2- 172.21.15.33:0/1526081324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 0x7f0ae8154ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80c2830 tx=0x7f0ae0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3eda9e080a4b854 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 0x7f5af0067970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54632/0 (socket says 172.21.15.33:54632) 2023-10-06T02:45:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 -- 172.21.15.33:0/2059730179 learned_addr learned my addr 172.21.15.33:0/2059730179 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601cbb9700 1 -- 172.21.15.33:0/690785463 >> v2:172.21.15.167:3300/0 conn(0x7f60180c2510 msgr2=0x7f60180ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601cbb9700 1 --2- 172.21.15.33:0/690785463 >> v2:172.21.15.167:3300/0 conn(0x7f60180c2510 0x7f60180ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601cbb9700 1 -- 172.21.15.33:0/690785463 >> v1:172.21.15.33:6789/0 conn(0x7f60180c9e60 legacy=0x7f6018068340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f6016ffd700 1 -- 172.21.15.33:0/690785463 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166297998 0 0) 0x7f60180c2080 con 0x7f60180c9e60 2023-10-06T02:45:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601cbb9700 1 -- 172.21.15.33:0/690785463 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180c1a30 con 0x7f60180c0eb0 2023-10-06T02:45:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f5706700 1 -- 172.21.15.33:0/3844551822 >> v1:172.21.15.167:6789/0 conn(0x7fd6f0161820 legacy=0x7fd6f00c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f5706700 1 -- 172.21.15.33:0/3844551822 >> v1:172.21.15.33:6789/0 conn(0x7fd6f01613a0 legacy=0x7fd6f00680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f5706700 1 -- 172.21.15.33:0/3844551822 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0067610 con 0x7fd6f0160fd0 2023-10-06T02:45:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af5a5f700 1 --2- 172.21.15.33:0/2059730179 >> v2:172.21.15.33:3300/0 conn(0x7f5af006d6e0 0x7f5af006dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f5706700 1 --2- 172.21.15.33:0/3844551822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f00c9550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009e60 tx=0x7fd6e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d16b2a945fe2a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0adffff700 1 -- 172.21.15.33:0/1526081324 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000e070 con 0x7f0ae806a520 2023-10-06T02:45:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 -- 172.21.15.33:0/2059730179 >> v2:172.21.15.33:3300/0 conn(0x7f5af006d6e0 msgr2=0x7f5af006dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 --2- 172.21.15.33:0/2059730179 >> v2:172.21.15.33:3300/0 conn(0x7f5af006d6e0 0x7f5af006dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 -- 172.21.15.33:0/2059730179 >> v1:172.21.15.33:6789/0 conn(0x7f5af00b12a0 legacy=0x7f5af00be560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 -- 172.21.15.33:0/2059730179 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af00c5fd0 con 0x7f5af00bec70 2023-10-06T02:45:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af5a5f700 1 --2- 172.21.15.33:0/2059730179 >> v2:172.21.15.33:3300/0 conn(0x7f5af006d6e0 0x7f5af006dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0adffff700 1 -- 172.21.15.33:0/1526081324 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae000ba60 con 0x7f0ae806a520 2023-10-06T02:45:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0adffff700 1 -- 172.21.15.33:0/1526081324 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000be00 con 0x7f0ae806a520 2023-10-06T02:45:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f601cbb9700 1 --2- 172.21.15.33:0/690785463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c0eb0 0x7f60181587f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f60080098e0 tx=0x7f60080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7464e791c7ac65d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800e070 con 0x7fd6f0160fd0 2023-10-06T02:45:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800bab0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6e7fff700 1 -- 172.21.15.33:0/3844551822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800be70 con 0x7fd6f0160fd0 2023-10-06T02:45:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6260700 1 --2- 172.21.15.33:0/2059730179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 0x7f5af0067970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009f20 tx=0x7f5ae8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd15b09441a3d792 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f6016ffd700 1 -- 172.21.15.33:0/690785463 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600800b640 con 0x7f60180c0eb0 2023-10-06T02:45:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f6016ffd700 1 -- 172.21.15.33:0/690785463 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600800b7a0 con 0x7f60180c0eb0 2023-10-06T02:45:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f6016ffd700 1 -- 172.21.15.33:0/690785463 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600800bb40 con 0x7f60180c0eb0 2023-10-06T02:45:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af4a5d700 1 -- 172.21.15.33:0/2059730179 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800e070 con 0x7f5af00bec70 2023-10-06T02:45:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af4a5d700 1 -- 172.21.15.33:0/2059730179 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae800bb10 con 0x7f5af00bec70 2023-10-06T02:45:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af4a5d700 1 -- 172.21.15.33:0/2059730179 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80092e0 con 0x7f5af00bec70 2023-10-06T02:45:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 -- 172.21.15.33:0/3844551822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 msgr2=0x7fd6f00c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 --2- 172.21.15.33:0/3844551822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f00c9550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009e60 tx=0x7fd6e8009780 comp rx=0 tx=0).stop 2023-10-06T02:45:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 -- 172.21.15.33:0/1526081324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 msgr2=0x7f0ae8154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 --2- 172.21.15.33:0/1526081324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 0x7f0ae8154ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80c2830 tx=0x7f0ae0009780 comp rx=0 tx=0).stop 2023-10-06T02:45:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 -- 172.21.15.33:0/1526081324 shutdown_connections 2023-10-06T02:45:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 --2- 172.21.15.33:0/1526081324 >> v2:172.21.15.33:3300/0 conn(0x7f0ae8154ff0 0x7f0ae80694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 --2- 172.21.15.33:0/1526081324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 0x7f0ae8154ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 -- 172.21.15.33:0/1526081324 >> 172.21.15.33:0/1526081324 conn(0x7f0ae80cc290 msgr2=0x7f0ae80c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 -- 172.21.15.33:0/1526081324 shutdown_connections 2023-10-06T02:45:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 -- 172.21.15.33:0/3844551822 shutdown_connections 2023-10-06T02:45:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 --2- 172.21.15.33:0/3844551822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f00c9550 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 -- 172.21.15.33:0/3844551822 >> 172.21.15.33:0/3844551822 conn(0x7fd6f00c83f0 msgr2=0x7fd6f00c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f0aee115700 1 -- 172.21.15.33:0/1526081324 wait complete. 2023-10-06T02:45:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 -- 172.21.15.33:0/3844551822 shutdown_connections 2023-10-06T02:45:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7fd6f796a700 1 -- 172.21.15.33:0/3844551822 wait complete. 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 -- 172.21.15.33:0/2059730179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 msgr2=0x7f5af0067970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 --2- 172.21.15.33:0/2059730179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 0x7f5af0067970 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009f20 tx=0x7f5ae8009840 comp rx=0 tx=0).stop 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 -- 172.21.15.33:0/2059730179 shutdown_connections 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 --2- 172.21.15.33:0/2059730179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af00bec70 0x7f5af0067970 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 --2- 172.21.15.33:0/2059730179 >> v2:172.21.15.33:3300/0 conn(0x7f5af006d6e0 0x7f5af006dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.209+0000 7f5af6a61700 1 -- 172.21.15.33:0/2059730179 >> 172.21.15.33:0/2059730179 conn(0x7f5af00cbb20 msgr2=0x7f5af00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 Processor -- start 2023-10-06T02:45:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 -- start start 2023-10-06T02:45:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 -- 172.21.15.33:0/690785463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c0eb0 msgr2=0x7f60181587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 --2- 172.21.15.33:0/690785463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c0eb0 0x7f60181587f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f60080098e0 tx=0x7f60080091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 Processor -- start 2023-10-06T02:45:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 -- start start 2023-10-06T02:45:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 Processor -- start 2023-10-06T02:45:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 -- start start 2023-10-06T02:45:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 -- 172.21.15.33:0/690785463 shutdown_connections 2023-10-06T02:45:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 --2- 172.21.15.33:0/690785463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c0eb0 0x7f60181587f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 --2- 172.21.15.33:0/690785463 >> v2:172.21.15.167:3300/0 conn(0x7f60180c2510 0x7f60180ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 -- 172.21.15.33:0/690785463 >> 172.21.15.33:0/690785463 conn(0x7f60180c83f0 msgr2=0x7f60180c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 Processor -- start 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f765806d6e0 0x7f765806dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f76581553f0 0x7f7658155870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76580b0b30 con 0x7f76580adf30 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7658069620 con 0x7f765806d6e0 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d993700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7658155db0 con 0x7f76581553f0 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f765806d6e0 0x7f765806dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f765806d6e0 0x7f765806dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54744/0 (socket says 172.21.15.33:54744) 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 -- 172.21.15.33:0/207886007 learned_addr learned my addr 172.21.15.33:0/207886007 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765d192700 1 --2- 172.21.15.33:0/207886007 >> v2:172.21.15.167:3300/0 conn(0x7f76581553f0 0x7f7658155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 -- 172.21.15.33:0/207886007 >> v2:172.21.15.167:3300/0 conn(0x7f76581553f0 msgr2=0x7f7658155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 --2- 172.21.15.33:0/207886007 >> v2:172.21.15.167:3300/0 conn(0x7f76581553f0 0x7f7658155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 -- 172.21.15.33:0/207886007 >> v1:172.21.15.33:6789/0 conn(0x7f76580adf30 legacy=0x7f76580b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 -- 172.21.15.33:0/207886007 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76580c5d20 con 0x7f765806d6e0 2023-10-06T02:45:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f765c991700 1 --2- 172.21.15.33:0/207886007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765806dab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76580af1a0 tx=0x7f7648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43a9ff6b8d0ba86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 0x7f0ae8149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 0x7f0ae80b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ae8154ff0 0x7f0ae80b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80afff0 con 0x7f0ae806e4f0 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae814a090 con 0x7f0ae806a520 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aee115700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80af370 con 0x7f0ae8154ff0 2023-10-06T02:45:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f5af6a61700 1 -- 172.21.15.33:0/2059730179 shutdown_connections 2023-10-06T02:45:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f76577fe700 1 -- 172.21.15.33:0/207886007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764800b820 con 0x7f765806d6e0 2023-10-06T02:45:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 -- start start 2023-10-06T02:45:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6f01613a0 0x7fd6f00af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6f0161820 0x7fd6f00af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00c4690 con 0x7fd6f0160fd0 2023-10-06T02:45:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00afca0 con 0x7fd6f0161820 2023-10-06T02:45:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f796a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00aff80 con 0x7fd6f01613a0 2023-10-06T02:45:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 0x7f0ae80b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 0x7f0ae80b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54748/0 (socket says 172.21.15.33:54748) 2023-10-06T02:45:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 -- 172.21.15.33:0/440630045 learned_addr learned my addr 172.21.15.33:0/440630045 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f76577fe700 1 -- 172.21.15.33:0/207886007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764800b980 con 0x7f765806d6e0 2023-10-06T02:45:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f76577fe700 1 -- 172.21.15.33:0/207886007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764800bd20 con 0x7f765806d6e0 2023-10-06T02:45:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f5af6a61700 1 -- 172.21.15.33:0/2059730179 wait complete. 2023-10-06T02:45:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54758/0 (socket says 172.21.15.33:54758) 2023-10-06T02:45:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 -- 172.21.15.33:0/2341934360 learned_addr learned my addr 172.21.15.33:0/2341934360 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7ff06fd49700 1 Processor -- start 2023-10-06T02:45:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f4f05700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6f01613a0 0x7fd6f00af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 -- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6f01613a0 msgr2=0x7fd6f00af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aed914700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ae8154ff0 0x7f0ae80b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 -- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ae8154ff0 msgr2=0x7f0ae80b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ae8154ff0 0x7f0ae80b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 -- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 msgr2=0x7f0ae8149b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T02:45:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 0x7f0ae8149b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 -- 172.21.15.33:0/440630045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009a20 con 0x7f0ae806e4f0 2023-10-06T02:45:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0aec912700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 0x7f0ae80b0780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ae806aab0 tx=0x7f0ae0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6f01613a0 0x7fd6f00af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 -- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6f0161820 msgr2=0x7fd6f00af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6f0161820 0x7fd6f00af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fd6f5706700 1 -- 172.21.15.33:0/2341934360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009a20 con 0x7fd6f0160fd0 2023-10-06T02:45:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0001090 con 0x7f0ae806e4f0 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0007080 con 0x7f0ae806e4f0 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00256b0 con 0x7f0ae806e4f0 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 0x7fe0e40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe0e40c2510 0x7fe0e4068340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e40c2080 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e40c1470 con 0x7fe0e40c2510 2023-10-06T02:45:44.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7fe0e8e28700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40c1750 con 0x7fe0e40c9e60 2023-10-06T02:45:44.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6f5706700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f014fbe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80099f0 tx=0x7fd6e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 -- 172.21.15.33:0/690785463 shutdown_connections 2023-10-06T02:45:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7f601e61c700 1 -- 172.21.15.33:0/690785463 wait complete. 2023-10-06T02:45:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8012d00 con 0x7fd6f0160fd0 2023-10-06T02:45:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e8012e80 con 0x7fd6f0160fd0 2023-10-06T02:45:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80276c0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6f796a700 1 -- 172.21.15.33:0/2341934360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f00b0260 con 0x7fd6f0160fd0 2023-10-06T02:45:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6f796a700 1 -- 172.21.15.33:0/2341934360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f00b0540 con 0x7fd6f0160fd0 2023-10-06T02:45:44.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 Processor -- start 2023-10-06T02:45:44.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 -- start start 2023-10-06T02:45:44.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 0x7fe0e40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 0x7fe0e40ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54650/0 (socket says 172.21.15.33:54650) 2023-10-06T02:45:44.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 -- 172.21.15.33:0/2340229138 learned_addr learned my addr 172.21.15.33:0/2340229138 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae80af650 con 0x7f0ae806e4f0 2023-10-06T02:45:44.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae80afc10 con 0x7f0ae806e4f0 2023-10-06T02:45:44.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 -- 172.21.15.33:0/207886007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 msgr2=0x7f765806dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 --2- 172.21.15.33:0/207886007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765806dab0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76580af1a0 tx=0x7f7648009250 comp rx=0 tx=0).stop 2023-10-06T02:45:44.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.213+0000 7ff06fd49700 1 -- start start 2023-10-06T02:45:44.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae81413b0 con 0x7f0ae806e4f0 2023-10-06T02:45:44.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6f4f05700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6f01613a0 0x7fd6f00af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:44.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e1d9b700 1 --2- 172.21.15.33:0/2340229138 >> v2:172.21.15.33:3300/0 conn(0x7fe0e40c2510 0x7fe0e4068340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 -- 172.21.15.33:0/207886007 shutdown_connections 2023-10-06T02:45:44.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 --2- 172.21.15.33:0/207886007 >> v2:172.21.15.167:3300/0 conn(0x7f76581553f0 0x7f7658155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 --2- 172.21.15.33:0/207886007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765806dab0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 -- 172.21.15.33:0/207886007 >> 172.21.15.33:0/207886007 conn(0x7f76580cbb20 msgr2=0x7f76580cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 Processor -- start 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 -- start start 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af006d6e0 0x7f5af00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 0x7f5af00c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af00bec70 0x7f5af0149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af00c4400 con 0x7f5af00b12a0 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af00c5240 con 0x7f5af00bec70 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6a61700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af015f880 con 0x7f5af006d6e0 2023-10-06T02:45:44.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 0x7f5af00c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 -- 172.21.15.33:0/2340229138 >> v2:172.21.15.33:3300/0 conn(0x7fe0e40c2510 msgr2=0x7fe0e4068340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 --2- 172.21.15.33:0/2340229138 >> v2:172.21.15.33:3300/0 conn(0x7fe0e40c2510 0x7fe0e4068340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 -- 172.21.15.33:0/2340229138 >> v1:172.21.15.33:6789/0 conn(0x7fe0e40c0eb0 legacy=0x7fe0e41587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 -- 172.21.15.33:0/2340229138 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e40c1a30 con 0x7fe0e40c9e60 2023-10-06T02:45:44.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 -- 172.21.15.33:0/207886007 shutdown_connections 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 0x7f5af00c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54780/0 (socket says 172.21.15.33:54780) 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 -- 172.21.15.33:0/534489797 learned_addr learned my addr 172.21.15.33:0/534489797 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af5a5f700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af006d6e0 0x7f5af00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6260700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af00bec70 0x7f5af0149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06fd49700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0680c0d10 0x7ff0680c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06fd49700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0680c99d0 con 0x7ff068160fd0 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06fd49700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff068067050 con 0x7ff0680c9550 2023-10-06T02:45:44.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06fd49700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff068067330 con 0x7ff0680c0d10 2023-10-06T02:45:44.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e259c700 1 --2- 172.21.15.33:0/2340229138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 0x7fe0e40ca230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40099b0 tx=0x7fe0d4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e1a795b5fc9ce5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06dae5700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff068160fd0 0x7ff0681613a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44736/0 (socket says 172.21.15.33:44736) 2023-10-06T02:45:44.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 -- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af006d6e0 msgr2=0x7f5af00c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af006d6e0 0x7f5af00c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 -- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af00bec70 msgr2=0x7f5af0149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af00bec70 0x7f5af0149550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 -- 172.21.15.33:0/534489797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009ae0 con 0x7f5af00b12a0 2023-10-06T02:45:44.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af6260700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af00bec70 0x7f5af0149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:44.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06dae5700 1 -- 172.21.15.33:0/1846473432 learned_addr learned my addr 172.21.15.33:0/1846473432 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:44.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e159a700 1 -- 172.21.15.33:0/2340229138 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400b860 con 0x7fe0e40c9e60 2023-10-06T02:45:44.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f5af525e700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 0x7f5af00c3ec0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c590 tx=0x7f5ae000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60180c0eb0 0x7f601814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c2510 0x7f60180a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60180c9e60 0x7f60181525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c4b60 con 0x7f60180c0eb0 2023-10-06T02:45:44.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c3f50 con 0x7f60180c9e60 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601e61c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c4230 con 0x7f60180c2510 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601cbb9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60180c9e60 0x7f60181525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601cbb9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60180c9e60 0x7f60181525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43778/0 (socket says 172.21.15.33:43778) 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f601cbb9700 1 -- 172.21.15.33:0/362268738 learned_addr learned my addr 172.21.15.33:0/362268738 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60180c0eb0 0x7f601814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fd6f796a700 1 -- 172.21.15.33:0/2341934360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0154cc0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f60177fe700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c2510 0x7f60180a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7ff06e2e6700 1 --2- 172.21.15.33:0/1846473432 >> v2:172.21.15.167:3300/0 conn(0x7ff0680c0d10 0x7ff0680c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f765d993700 1 -- 172.21.15.33:0/207886007 wait complete. 2023-10-06T02:45:44.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 -- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c2510 msgr2=0x7f60180a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7fe0e159a700 1 -- 172.21.15.33:0/2340229138 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d400b9c0 con 0x7fe0e40c9e60 2023-10-06T02:45:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c2510 0x7f60180a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 -- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60180c9e60 msgr2=0x7f60181525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60180c9e60 0x7f60181525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 -- 172.21.15.33:0/362268738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008009420 con 0x7f60180c0eb0 2023-10-06T02:45:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.217+0000 7f6017fff700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60180c0eb0 0x7f601814c360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f601000cc10 tx=0x7f6010001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e159a700 1 -- 172.21.15.33:0/2340229138 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400bd60 con 0x7fe0e40c9e60 2023-10-06T02:45:44.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06e2e6700 1 -- 172.21.15.33:0/1846473432 >> v1:172.21.15.167:6789/0 conn(0x7ff0680c9550 legacy=0x7ff0680680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06e2e6700 1 -- 172.21.15.33:0/1846473432 >> v1:172.21.15.33:6789/0 conn(0x7ff068160fd0 legacy=0x7ff0681613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06e2e6700 1 -- 172.21.15.33:0/1846473432 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068067610 con 0x7ff0680c0d10 2023-10-06T02:45:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2340229138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 msgr2=0x7fe0e40ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2340229138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 0x7fe0e40ca230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40099b0 tx=0x7fe0d4009250 comp rx=0 tx=0).stop 2023-10-06T02:45:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06e2e6700 1 --2- 172.21.15.33:0/1846473432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c0d10 0x7ff0680c1170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff0640098e0 tx=0x7ff0640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=917d384da2044fae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06cae3700 1 -- 172.21.15.33:0/1846473432 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064009e80 con 0x7ff0680c0d10 2023-10-06T02:45:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0ae0025aa0 con 0x7f0ae806e4f0 2023-10-06T02:45:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f0add7fa700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0acc0606a0 0x7f0acc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0ae007f250 con 0x7f0ae806e4f0 2023-10-06T02:45:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06cae3700 1 -- 172.21.15.33:0/1846473432 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06400b860 con 0x7ff0680c0d10 2023-10-06T02:45:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06cae3700 1 -- 172.21.15.33:0/1846473432 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400bbe0 con 0x7ff0680c0d10 2023-10-06T02:45:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 Processor -- start 2023-10-06T02:45:44.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2340229138 shutdown_connections 2023-10-06T02:45:44.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2340229138 >> v2:172.21.15.33:3300/0 conn(0x7fe0e40c2510 0x7fe0e4068340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2340229138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c9e60 0x7fe0e40ca230 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2340229138 >> 172.21.15.33:0/2340229138 conn(0x7fe0e40c83f0 msgr2=0x7fe0e40c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 -- start start 2023-10-06T02:45:44.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f0aed113700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0acc0606a0 0x7f0acc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd6e8024070 con 0x7fd6f0160fd0 2023-10-06T02:45:44.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 -- 172.21.15.33:0/1846473432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c0d10 msgr2=0x7ff0680c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 --2- 172.21.15.33:0/1846473432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c0d10 0x7ff0680c1170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff0640098e0 tx=0x7ff0640091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fd6e57fa700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6d4060620 0x7fd6d4062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 -- 172.21.15.33:0/1846473432 shutdown_connections 2023-10-06T02:45:44.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 --2- 172.21.15.33:0/1846473432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c0d10 0x7ff0680c1170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 -- 172.21.15.33:0/1846473432 >> 172.21.15.33:0/1846473432 conn(0x7ff0680c83f0 msgr2=0x7ff0680c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f0aed113700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0acc0606a0 0x7f0acc062b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8007fd0 tx=0x7f0ad8007a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd6e807f0b0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 -- 172.21.15.33:0/1846473432 shutdown_connections 2023-10-06T02:45:44.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2340229138 shutdown_connections 2023-10-06T02:45:44.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 -- 172.21.15.33:0/1846473432 wait complete. 2023-10-06T02:45:44.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fd6f4f05700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6d4060620 0x7fd6d4062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2340229138 wait complete. 2023-10-06T02:45:44.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765816a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580adf30 0x7f765814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76581553f0 0x7f76580c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f033e3ba700 1 Processor -- start 2023-10-06T02:45:44.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c28c0 con 0x7f765806d6e0 2023-10-06T02:45:44.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c2ba0 con 0x7f76581553f0 2023-10-06T02:45:44.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765d993700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c2e80 con 0x7f76580adf30 2023-10-06T02:45:44.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765816a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765816a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54786/0 (socket says 172.21.15.33:54786) 2023-10-06T02:45:44.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 -- 172.21.15.33:0/3314573604 learned_addr learned my addr 172.21.15.33:0/3314573604 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f033e3ba700 1 -- start start 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f7657fff700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580adf30 0x7f765814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 -- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580adf30 msgr2=0x7f765814ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580adf30 0x7f765814ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 -- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76581553f0 msgr2=0x7f76580c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76581553f0 0x7f76580c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 -- 172.21.15.33:0/3314573604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480094f0 con 0x7f765806d6e0 2023-10-06T02:45:44.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7f765c991700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765816a080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7648003080 tx=0x7f7648016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 Processor -- start 2023-10-06T02:45:44.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.221+0000 7ff06fd49700 1 -- start start 2023-10-06T02:45:44.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 Processor -- start 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0337fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0338158b00 0x7f0338067970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44746/0 (socket says 172.21.15.33:44746) 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f03380ca170 0x7f033815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f0338158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380c0f20 con 0x7f0338158b00 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03380c2660 con 0x7f03380ca170 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0338068080 con 0x7f033815cd30 2023-10-06T02:45:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 -- start start 2023-10-06T02:45:44.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0337fff700 1 -- 172.21.15.33:0/2858047865 learned_addr learned my addr 172.21.15.33:0/2858047865 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:44.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fd6f4f05700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6d4060620 0x7fd6d4062ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00c4f70 tx=0x7fd6ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06fd49700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0680c0d10 0x7ff06815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06fd49700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c9550 0x7ff0680b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06fd49700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff068160fd0 0x7ff0680afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06fd49700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680af620 con 0x7ff0680c0d10 2023-10-06T02:45:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06fd49700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06815e6a0 con 0x7ff0680c9550 2023-10-06T02:45:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06fd49700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680aea10 con 0x7ff068160fd0 2023-10-06T02:45:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0011590 con 0x7f5af00b12a0 2023-10-06T02:45:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648011e20 con 0x7f765806d6e0 2023-10-06T02:45:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000a850 con 0x7f60180c0eb0 2023-10-06T02:45:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76580c3160 con 0x7f765806d6e0 2023-10-06T02:45:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f5af6a61700 1 -- 172.21.15.33:0/534489797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af00c4630 con 0x7f5af00b12a0 2023-10-06T02:45:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765816a5c0 con 0x7f765806d6e0 2023-10-06T02:45:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c9550 0x7ff0680b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06dae5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0680c0d10 0x7ff06815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06dae5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0680c0d10 0x7ff06815e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54788/0 (socket says 172.21.15.33:54788) 2023-10-06T02:45:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06dae5700 1 -- 172.21.15.33:0/4000473363 learned_addr learned my addr 172.21.15.33:0/4000473363 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f5af6a61700 1 -- 172.21.15.33:0/534489797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af00c4bf0 con 0x7f5af00b12a0 2023-10-06T02:45:44.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f03377fe700 1 --2- 172.21.15.33:0/2858047865 >> v2:172.21.15.167:3300/0 conn(0x7f03380ca170 0x7f033815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033c957700 1 --2- 172.21.15.33:0/2858047865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f0338158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0336ffd700 1 -- 172.21.15.33:0/2858047865 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678706618 0 0) 0x7f03380c0f20 con 0x7f0338158b00 2023-10-06T02:45:44.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0336ffd700 1 -- 172.21.15.33:0/2858047865 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0320003570 con 0x7f0338158b00 2023-10-06T02:45:44.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60180c4510 con 0x7f60180c0eb0 2023-10-06T02:45:44.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601814c8a0 con 0x7f60180c0eb0 2023-10-06T02:45:44.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033c957700 1 -- 172.21.15.33:0/2858047865 >> v2:172.21.15.167:3300/0 conn(0x7f03380ca170 msgr2=0x7f033815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:44.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033c957700 1 --2- 172.21.15.33:0/2858047865 >> v2:172.21.15.167:3300/0 conn(0x7f03380ca170 0x7f033815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033c957700 1 -- 172.21.15.33:0/2858047865 >> v1:172.21.15.33:6789/0 conn(0x7f0338158b00 legacy=0x7f0338067970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033c957700 1 -- 172.21.15.33:0/2858047865 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380c6000 con 0x7f033815cd30 2023-10-06T02:45:44.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033c957700 1 --2- 172.21.15.33:0/2858047865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f0338158ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098e0 tx=0x7f032c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c923d2e1795e07f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0336ffd700 1 -- 172.21.15.33:0/2858047865 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00b640 con 0x7f033815cd30 2023-10-06T02:45:44.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 -- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff068160fd0 msgr2=0x7ff0680afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff068160fd0 0x7ff0680afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 -- 172.21.15.33:0/4000473363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0680c0d10 msgr2=0x7ff06815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0680c0d10 0x7ff06815e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 -- 172.21.15.33:0/4000473363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009420 con 0x7ff0680c9550 2023-10-06T02:45:44.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0336ffd700 1 -- 172.21.15.33:0/2858047865 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c00b7a0 con 0x7f033815cd30 2023-10-06T02:45:44.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0336ffd700 1 -- 172.21.15.33:0/2858047865 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00bb20 con 0x7f033815cd30 2023-10-06T02:45:44.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f5af6a61700 1 -- 172.21.15.33:0/534489797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af0071040 con 0x7f5af00b12a0 2023-10-06T02:45:44.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7ff06d2e4700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c9550 0x7ff0680b0eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff05800c590 tx=0x7ff058007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76580c33a0 con 0x7f765806d6e0 2023-10-06T02:45:44.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60180710b0 con 0x7f60180c0eb0 2023-10-06T02:45:44.282 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 0x7fe0e414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c2510 0x7fe0e40ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e40c9e60 0x7fe0e40c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40c59c0 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40a93a0 con 0x7fe0e40c9e60 2023-10-06T02:45:44.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e8e28700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40a8a70 con 0x7fe0e40c2510 2023-10-06T02:45:44.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0ae004f450 con 0x7f0ae806e4f0 2023-10-06T02:45:44.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 0x7fe0e414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 0x7fe0e414c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54790/0 (socket says 172.21.15.33:54790) 2023-10-06T02:45:44.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 -- 172.21.15.33:0/2829275860 learned_addr learned my addr 172.21.15.33:0/2829275860 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 -- 172.21.15.33:0/2858047865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 msgr2=0x7f0338158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7f033e3ba700 1 --2- 172.21.15.33:0/2858047865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f0338158ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098e0 tx=0x7f032c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e1d9b700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c2510 0x7fe0e40ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e2d9d700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e40c9e60 0x7fe0e40c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 -- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c2510 msgr2=0x7fe0e40ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c2510 0x7fe0e40ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 -- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e40c9e60 msgr2=0x7fe0e40c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e40c9e60 0x7fe0e40c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 -- 172.21.15.33:0/2829275860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d40094f0 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.225+0000 7fe0e259c700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 0x7fe0e414c740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40c1890 tx=0x7fe0d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058011590 con 0x7ff0680c9550 2023-10-06T02:45:44.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff058011710 con 0x7ff0680c9550 2023-10-06T02:45:44.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0680aecf0 con 0x7ff0680c9550 2023-10-06T02:45:44.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058011ad0 con 0x7ff0680c9550 2023-10-06T02:45:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0680af2b0 con 0x7ff0680c9550 2023-10-06T02:45:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff05801b3b0 con 0x7ff0680c9550 2023-10-06T02:45:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff05dffb700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff0540606e0 0x7ff054062b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff05807c0c0 con 0x7ff0680c9550 2023-10-06T02:45:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff06dae5700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff0540606e0 0x7ff054062b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.229+0000 7ff06dae5700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff0540606e0 0x7ff054062b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff060002250 tx=0x7ff060009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- 172.21.15.33:0/2858047865 shutdown_connections 2023-10-06T02:45:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 --2- 172.21.15.33:0/2858047865 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f0338158ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 --2- 172.21.15.33:0/2858047865 >> v2:172.21.15.167:3300/0 conn(0x7f03380ca170 0x7f033815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- 172.21.15.33:0/2858047865 >> 172.21.15.33:0/2858047865 conn(0x7f03380c8250 msgr2=0x7f03380c8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- 172.21.15.33:0/2858047865 shutdown_connections 2023-10-06T02:45:44.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- 172.21.15.33:0/2858047865 wait complete. 2023-10-06T02:45:44.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6f0154cc0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0011900 con 0x7f5af00b12a0 2023-10-06T02:45:44.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0011a70 con 0x7f5af00b12a0 2023-10-06T02:45:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764801b890 con 0x7f765806d6e0 2023-10-06T02:45:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764801ba40 con 0x7f765806d6e0 2023-10-06T02:45:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d401a450 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e40a8d50 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e40c4230 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601000abc0 con 0x7f60180c0eb0 2023-10-06T02:45:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000ad30 con 0x7f60180c0eb0 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 Processor -- start 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- start start 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03380ca170 0x7f0338147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0338158b00 0x7f03380c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f03380c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380c56f0 con 0x7f0338158b00 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380c5b80 con 0x7f03380ca170 2023-10-06T02:45:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033e3ba700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380c4840 con 0x7f033815cd30 2023-10-06T02:45:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d401a7c0 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d401a930 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f0337fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03380ca170 0x7f0338147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f0337fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03380ca170 0x7f0338147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43810/0 (socket says 172.21.15.33:43810) 2023-10-06T02:45:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f0337fff700 1 -- 172.21.15.33:0/777955792 learned_addr learned my addr 172.21.15.33:0/777955792 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f03380c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6010025430 con 0x7f60180c0eb0 2023-10-06T02:45:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 -- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03380ca170 msgr2=0x7f0338147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03380ca170 0x7f0338147450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 -- 172.21.15.33:0/777955792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0338158b00 msgr2=0x7f03380c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0338158b00 0x7f03380c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 -- 172.21.15.33:0/777955792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009420 con 0x7f033815cd30 2023-10-06T02:45:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f5ffffff700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6004060550 0x7f6004062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.233+0000 7f033c957700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f03380c6950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f032c0093f0 tx=0x7f032c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f601007d830 con 0x7f60180c0eb0 2023-10-06T02:45:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe0d401aa90 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f60177fe700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6004060550 0x7f6004062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5ae0011bd0 con 0x7f5af00b12a0 2023-10-06T02:45:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7fe0d27fc700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe0cc0606a0 0x7fe0cc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7fe0e1d9b700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe0cc0606a0 0x7fe0cc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f5ae5ffb700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5ad4060580 0x7f5ad4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f5ae007c6b0 con 0x7f5af00b12a0 2023-10-06T02:45:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f60177fe700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6004060550 0x7f6004062a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6018154c20 tx=0x7f600c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f764801bba0 con 0x7f765806d6e0 2023-10-06T02:45:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f5af5a5f700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5ad4060580 0x7f5ad4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f7654ff9700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7644060550 0x7f7644062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f764807e760 con 0x7f765806d6e0 2023-10-06T02:45:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f7657fff700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7644060550 0x7f7644062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7fe0e1d9b700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe0cc0606a0 0x7fe0cc062b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40ad580 tx=0x7fe0d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f5af5a5f700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5ad4060580 0x7f5ad4062a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006f00 tx=0x7f5aec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.237+0000 7f7657fff700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7644060550 0x7f7644062a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76581589d0 tx=0x7f764c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00b9e0 con 0x7f033815cd30 2023-10-06T02:45:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c026480 con 0x7f033815cd30 2023-10-06T02:45:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03380c4ae0 con 0x7f033815cd30 2023-10-06T02:45:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c80052b0 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0680710b0 con 0x7ff0680c9550 2023-10-06T02:45:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03380c50a0 con 0x7f033815cd30 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c026820 con 0x7f033815cd30 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f032c023030 con 0x7f033815cd30 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0327fff700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f031c060620 0x7f031c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f032c07e220 con 0x7f033815cd30 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0337fff700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f031c060620 0x7f031c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.241+0000 7f0337fff700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f031c060620 0x7f031c062ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0328001ad0 tx=0x7f0328009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.245+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ae004c900 con 0x7f5af00b12a0 2023-10-06T02:45:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.249+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03140052b0 con 0x7f033815cd30 2023-10-06T02:45:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.249+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff058017080 con 0x7ff0680c9550 2023-10-06T02:45:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.253+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f601004da50 con 0x7f60180c0eb0 2023-10-06T02:45:44.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.257+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f032c04e560 con 0x7f033815cd30 2023-10-06T02:45:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.261+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f764804e980 con 0x7f765806d6e0 2023-10-06T02:45:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.261+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe0d4052a40 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.269+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe0d404e640 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.553+0000 7fd6f796a700 1 -- 172.21.15.33:0/2341934360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd6f0154cc0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.553+0000 7fd6e57fa700 1 -- 172.21.15.33:0/2341934360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd6f0154cc0 con 0x7fd6f0160fd0 2023-10-06T02:45:44.562 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 -- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6d4060620 msgr2=0x7fd6d4062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6d4060620 0x7fd6d4062ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00c4f70 tx=0x7fd6ec009040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 -- 172.21.15.33:0/2341934360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 msgr2=0x7fd6f014fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f014fbe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80099f0 tx=0x7fd6e8019000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7f5af6a61700 1 -- 172.21.15.33:0/534489797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5af00b0b40 con 0x7f5af00b12a0 2023-10-06T02:45:44.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 -- 172.21.15.33:0/2341934360 shutdown_connections 2023-10-06T02:45:44.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd6d4060620 0x7fd6d4062ad0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6f01613a0 0x7fd6f00af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6f0161820 0x7fd6f00af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 --2- 172.21.15.33:0/2341934360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f0160fd0 0x7fd6f014fbe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 -- 172.21.15.33:0/2341934360 >> 172.21.15.33:0/2341934360 conn(0x7fd6f00c83f0 msgr2=0x7fd6f00c8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7f5ae5ffb700 1 -- 172.21.15.33:0/534489797 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ae0017090 con 0x7f5af00b12a0 2023-10-06T02:45:44.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 -- 172.21.15.33:0/2341934360 shutdown_connections 2023-10-06T02:45:44.568 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.561+0000 7fd6daffd700 1 -- 172.21.15.33:0/2341934360 wait complete. 2023-10-06T02:45:44.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0ae8071040 con 0x7f0ae806e4f0 2023-10-06T02:45:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 -- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5ad4060580 msgr2=0x7f5ad4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5ad4060580 0x7f5ad4062a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006f00 tx=0x7f5aec008040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 -- 172.21.15.33:0/534489797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 msgr2=0x7f5af00c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 0x7f5af00c3ec0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c590 tx=0x7f5ae000c8a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f0add7fa700 1 -- 172.21.15.33:0/440630045 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ae0052a70 con 0x7f0ae806e4f0 2023-10-06T02:45:44.574 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 -- 172.21.15.33:0/534489797 shutdown_connections 2023-10-06T02:45:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af00bec70 0x7f5af0149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5af00b12a0 0x7f5af00c3ec0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5ad4060580 0x7f5ad4062a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 --2- 172.21.15.33:0/534489797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5af006d6e0 0x7f5af00c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 -- 172.21.15.33:0/534489797 >> 172.21.15.33:0/534489797 conn(0x7f5af00cbb20 msgr2=0x7f5af00afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 -- 172.21.15.33:0/534489797 shutdown_connections 2023-10-06T02:45:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.569+0000 7f5adb7fe700 1 -- 172.21.15.33:0/534489797 wait complete. 2023-10-06T02:45:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0acc0606a0 msgr2=0x7f0acc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0acc0606a0 0x7f0acc062b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8007fd0 tx=0x7f0ad8007a50 comp rx=0 tx=0).stop 2023-10-06T02:45:44.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 msgr2=0x7f0ae80b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 0x7f0ae80b0780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ae806aab0 tx=0x7f0ae0018000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 shutdown_connections 2023-10-06T02:45:44.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0acc0606a0 0x7f0acc062b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.3 2023-10-06T02:45:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ae8154ff0 0x7f0ae80b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ae806e4f0 0x7f0ae80b0780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 --2- 172.21.15.33:0/440630045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ae806a520 0x7f0ae8149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 >> 172.21.15.33:0/440630045 conn(0x7f0ae80cc290 msgr2=0x7f0ae8069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0314005830 con 0x7f033815cd30 2023-10-06T02:45:44.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 shutdown_connections 2023-10-06T02:45:44.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0aee115700 1 -- 172.21.15.33:0/440630045 wait complete. 2023-10-06T02:45:44.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.573+0000 7f0327fff700 1 -- 172.21.15.33:0/777955792 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f032c015020 con 0x7f033815cd30 2023-10-06T02:45:44.583 INFO:teuthology.orchestra.run.smithi033.stdout:60129542151 2023-10-06T02:45:44.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.577+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f60180c1d10 con 0x7f60180c0eb0 2023-10-06T02:45:44.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.577+0000 7f5ffffff700 1 -- 172.21.15.33:0/362268738 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6010051070 con 0x7f60180c0eb0 2023-10-06T02:45:44.584 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f031c060620 msgr2=0x7f031c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f031c060620 0x7f031c062ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0328001ad0 tx=0x7f0328009040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 msgr2=0x7f03380c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f03380c6950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f032c0093f0 tx=0x7f032c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 shutdown_connections 2023-10-06T02:45:44.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f031c060620 0x7f031c062ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f033815cd30 0x7f03380c6950 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0338158b00 0x7f03380c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 --2- 172.21.15.33:0/777955792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03380ca170 0x7f0338147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 >> 172.21.15.33:0/777955792 conn(0x7f03380c8250 msgr2=0x7f03380c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 shutdown_connections 2023-10-06T02:45:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f033e3ba700 1 -- 172.21.15.33:0/777955792 wait complete. 2023-10-06T02:45:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6004060550 msgr2=0x7f6004062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6004060550 0x7f6004062a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6018154c20 tx=0x7f600c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60180c0eb0 msgr2=0x7f601814c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60180c0eb0 0x7f601814c360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f601000cc10 tx=0x7f6010001970 comp rx=0 tx=0).stop 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 shutdown_connections 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6004060550 0x7f6004062a00 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6018154c20 tx=0x7f600c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60180c2510 0x7f60180a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60180c9e60 0x7f60181525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 --2- 172.21.15.33:0/362268738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60180c0eb0 0x7f601814c360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 >> 172.21.15.33:0/362268738 conn(0x7f60180c83f0 msgr2=0x7f6018069340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff0680af760 con 0x7ff0680c9550 2023-10-06T02:45:44.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.581+0000 7ff05dffb700 1 -- 172.21.15.33:0/4000473363 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff05804c2e0 con 0x7ff0680c9550 2023-10-06T02:45:44.589 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.585+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 shutdown_connections 2023-10-06T02:45:44.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.585+0000 7f601e61c700 1 -- 172.21.15.33:0/362268738 wait complete. 2023-10-06T02:45:44.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.7 2023-10-06T02:45:44.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.4 2023-10-06T02:45:44.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.585+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f76580b0b30 con 0x7f765806d6e0 2023-10-06T02:45:44.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.585+0000 7f7654ff9700 1 -- 172.21.15.33:0/3314573604 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76580b0b30 con 0x7f765806d6e0 2023-10-06T02:45:44.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff0540606e0 msgr2=0x7ff054062b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff0540606e0 0x7ff054062b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff060002250 tx=0x7ff060009040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c9550 msgr2=0x7ff0680b0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c9550 0x7ff0680b0eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff05800c590 tx=0x7ff058007ec0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.593 INFO:teuthology.orchestra.run.smithi033.stdout:60129542152 2023-10-06T02:45:44.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 shutdown_connections 2023-10-06T02:45:44.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff0540606e0 0x7ff054062b90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff068160fd0 0x7ff0680afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0680c9550 0x7ff0680b0eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 --2- 172.21.15.33:0/4000473363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0680c0d10 0x7ff06815e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 >> 172.21.15.33:0/4000473363 conn(0x7ff0680c83f0 msgr2=0x7ff0680c78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 shutdown_connections 2023-10-06T02:45:44.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7ff06fd49700 1 -- 172.21.15.33:0/4000473363 wait complete. 2023-10-06T02:45:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7644060550 msgr2=0x7f7644062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7644060550 0x7f7644062a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76581589d0 tx=0x7f764c009040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 msgr2=0x7f765816a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765816a080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7648003080 tx=0x7f7648016000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 shutdown_connections 2023-10-06T02:45:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7644060550 0x7f7644062a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76580adf30 0x7f765814ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76581553f0 0x7f76580c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 --2- 172.21.15.33:0/3314573604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765806d6e0 0x7f765816a080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 >> 172.21.15.33:0/3314573604 conn(0x7f76580cbb20 msgr2=0x7f765806dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 shutdown_connections 2023-10-06T02:45:44.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.589+0000 7f765d993700 1 -- 172.21.15.33:0/3314573604 wait complete. 2023-10-06T02:45:44.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.593+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe0c8005560 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.593+0000 7fe0d27fc700 1 -- 172.21.15.33:0/2829275860 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0d4015030 con 0x7fe0e40c0eb0 2023-10-06T02:45:44.599 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:44.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.0 2023-10-06T02:45:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe0cc0606a0 msgr2=0x7fe0cc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe0cc0606a0 0x7fe0cc062b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40ad580 tx=0x7fe0d8008040 comp rx=0 tx=0).stop 2023-10-06T02:45:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 msgr2=0x7fe0e414c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 0x7fe0e414c740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40c1890 tx=0x7fe0d4017000 comp rx=0 tx=0).stop 2023-10-06T02:45:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 shutdown_connections 2023-10-06T02:45:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe0cc0606a0 0x7fe0cc062b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0e40c2510 0x7fe0e40ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e40c9e60 0x7fe0e40c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 --2- 172.21.15.33:0/2829275860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0e40c0eb0 0x7fe0e414c740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:44.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 >> 172.21.15.33:0/2829275860 conn(0x7fe0e40c83f0 msgr2=0x7fe0e415d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:44.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 shutdown_connections 2023-10-06T02:45:44.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:44.597+0000 7fe0e8e28700 1 -- 172.21.15.33:0/2829275860 wait complete. 2023-10-06T02:45:44.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.2 2023-10-06T02:45:44.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.1 2023-10-06T02:45:44.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.5 2023-10-06T02:45:44.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.6 2023-10-06T02:45:45.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T02:45:45.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T02:45:45.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T02:45:45.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T02:45:45.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T02:45:45.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T02:45:45.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T02:45:45.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T02:45:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.725+0000 7f8b6a655700 1 Processor -- start 2023-10-06T02:45:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.725+0000 7f8b6a655700 1 -- start start 2023-10-06T02:45:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154ab0 0x7f8b6406a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8b640694d0 0x7f8b64069950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b640c79c0 con 0x7f8b64154e80 2023-10-06T02:45:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b64069e90 con 0x7f8b640694d0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640c7e50 con 0x7f8b64154ab0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b68e52700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8b64154e80 0x7f8b640c65b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44758/0 (socket says 172.21.15.33:44758) 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b68e52700 1 -- 172.21.15.33:0/2211472013 learned_addr learned my addr 172.21.15.33:0/2211472013 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69653700 1 --2- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154ab0 0x7f8b6406a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69e54700 1 --2- 172.21.15.33:0/2211472013 >> v2:172.21.15.167:3300/0 conn(0x7f8b640694d0 0x7f8b64069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69e54700 1 -- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154ab0 msgr2=0x7f8b6406a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69e54700 1 --2- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154ab0 0x7f8b6406a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69e54700 1 -- 172.21.15.33:0/2211472013 >> v1:172.21.15.33:6789/0 conn(0x7f8b64154e80 legacy=0x7f8b640c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69e54700 1 -- 172.21.15.33:0/2211472013 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640c6db0 con 0x7f8b640694d0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b69e54700 1 --2- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b640694d0 0x7f8b64069950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000dd10 tx=0x7f8b6000d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=161ec23e224a3843 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b5bfff700 1 -- 172.21.15.33:0/2211472013 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60012070 con 0x7f8b640694d0 2023-10-06T02:45:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b5bfff700 1 -- 172.21.15.33:0/2211472013 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6000de70 con 0x7f8b640694d0 2023-10-06T02:45:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 -- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b640694d0 msgr2=0x7f8b64069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.729+0000 7f8b6a655700 1 --2- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b640694d0 0x7f8b64069950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000dd10 tx=0x7f8b6000d630 comp rx=0 tx=0).stop 2023-10-06T02:45:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- 172.21.15.33:0/2211472013 shutdown_connections 2023-10-06T02:45:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 --2- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b640694d0 0x7f8b64069950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 --2- 172.21.15.33:0/2211472013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154ab0 0x7f8b6406a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- 172.21.15.33:0/2211472013 >> 172.21.15.33:0/2211472013 conn(0x7f8b640cc290 msgr2=0x7f8b640c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- 172.21.15.33:0/2211472013 shutdown_connections 2023-10-06T02:45:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- 172.21.15.33:0/2211472013 wait complete. 2023-10-06T02:45:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 Processor -- start 2023-10-06T02:45:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- start start 2023-10-06T02:45:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b640694d0 0x7f8b640c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b64154ab0 0x7f8b64151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154e80 0x7f8b641565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640c4220 con 0x7f8b640694d0 2023-10-06T02:45:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640c3850 con 0x7f8b64154ab0 2023-10-06T02:45:45.740 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.733+0000 7f8b6a655700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640c3230 con 0x7f8b64154e80 2023-10-06T02:45:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b69653700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b640694d0 0x7f8b640c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b69653700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b640694d0 0x7f8b640c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54812/0 (socket says 172.21.15.33:54812) 2023-10-06T02:45:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b69653700 1 -- 172.21.15.33:0/2404965150 learned_addr learned my addr 172.21.15.33:0/2404965150 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b64154ab0 0x7f8b64151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b69e54700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154e80 0x7f8b641565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 -- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154e80 msgr2=0x7f8b641565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154e80 0x7f8b641565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 -- 172.21.15.33:0/2404965150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b640694d0 msgr2=0x7f8b640c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b640694d0 0x7f8b640c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 -- 172.21.15.33:0/2404965150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6000d8d0 con 0x7f8b64154ab0 2023-10-06T02:45:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b68e52700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b64154ab0 0x7f8b64151200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c5c0 tx=0x7f8b54007e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54005b30 con 0x7f8b64154ab0 2023-10-06T02:45:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54005cb0 con 0x7f8b64154ab0 2023-10-06T02:45:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54007320 con 0x7f8b64154ab0 2023-10-06T02:45:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b6a655700 1 -- 172.21.15.33:0/2404965150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b64159c50 con 0x7f8b64154ab0 2023-10-06T02:45:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.737+0000 7f8b6a655700 1 -- 172.21.15.33:0/2404965150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b64151740 con 0x7f8b64154ab0 2023-10-06T02:45:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.741+0000 7f8b6a655700 1 -- 172.21.15.33:0/2404965150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b400052b0 con 0x7f8b64154ab0 2023-10-06T02:45:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.741+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8b5401b3b0 con 0x7f8b64154ab0 2023-10-06T02:45:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.741+0000 7f8b597fa700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b480605d0 0x7f8b48062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.741+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f8b5407bf90 con 0x7f8b64154ab0 2023-10-06T02:45:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.741+0000 7f8b69653700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b480605d0 0x7f8b48062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.741+0000 7f8b69653700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b480605d0 0x7f8b48062a80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c008160 tx=0x7f8b5c00d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.753+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8b5404c090 con 0x7f8b64154ab0 2023-10-06T02:45:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 Processor -- start 2023-10-06T02:45:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 -- start start 2023-10-06T02:45:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8730154ab0 0x7f873006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8730154e80 0x7f87300c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300c79c0 con 0x7f87300694d0 2023-10-06T02:45:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87300c7e50 con 0x7f8730154e80 2023-10-06T02:45:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f873736b700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87300c6af0 con 0x7f8730154ab0 2023-10-06T02:45:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f614db50700 1 Processor -- start 2023-10-06T02:45:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f8736b6a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87300694d0 0x7f8730158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44770/0 (socket says 172.21.15.33:44770) 2023-10-06T02:45:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f8736369700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8730154ab0 0x7f873006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f8735b68700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8730154e80 0x7f87300c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f8735b68700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8730154e80 0x7f87300c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54826/0 (socket says 172.21.15.33:54826) 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f8735b68700 1 -- 172.21.15.33:0/1716433268 learned_addr learned my addr 172.21.15.33:0/1716433268 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735367700 1 -- 172.21.15.33:0/1716433268 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657880189 0 0) 0x7f87300c79c0 con 0x7f87300694d0 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735367700 1 -- 172.21.15.33:0/1716433268 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f871c003570 con 0x7f87300694d0 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735b68700 1 -- 172.21.15.33:0/1716433268 >> v2:172.21.15.167:3300/0 conn(0x7f8730154ab0 msgr2=0x7f873006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735b68700 1 --2- 172.21.15.33:0/1716433268 >> v2:172.21.15.167:3300/0 conn(0x7f8730154ab0 0x7f873006a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735b68700 1 -- 172.21.15.33:0/1716433268 >> v1:172.21.15.33:6789/0 conn(0x7f87300694d0 legacy=0x7f8730158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735b68700 1 -- 172.21.15.33:0/1716433268 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300c6db0 con 0x7f8730154e80 2023-10-06T02:45:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.761+0000 7f614db50700 1 -- start start 2023-10-06T02:45:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f614814f290 0x7f61480c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61480c2820 con 0x7f614814f660 2023-10-06T02:45:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61480c1c10 con 0x7f61480c1680 2023-10-06T02:45:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61480c1ef0 con 0x7f614814f290 2023-10-06T02:45:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61477fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f614814f290 0x7f61480c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f6147fff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f61480c1680 0x7f61480ad550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33714/0 (socket says 172.21.15.33:33714) 2023-10-06T02:45:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f6147fff700 1 -- 172.21.15.33:0/826323109 learned_addr learned my addr 172.21.15.33:0/826323109 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735b68700 1 --2- 172.21.15.33:0/1716433268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f87300c65b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f873005e9b0 tx=0x7f872c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdc5a5197ed983b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735367700 1 -- 172.21.15.33:0/1716433268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00b640 con 0x7f8730154e80 2023-10-06T02:45:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735367700 1 -- 172.21.15.33:0/1716433268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00b7a0 con 0x7f8730154e80 2023-10-06T02:45:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f8735367700 1 -- 172.21.15.33:0/1716433268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00bb60 con 0x7f8730154e80 2023-10-06T02:45:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61477fe700 1 -- 172.21.15.33:0/826323109 >> v1:172.21.15.167:6789/0 conn(0x7f61480c1680 legacy=0x7f61480ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61477fe700 1 -- 172.21.15.33:0/826323109 >> v1:172.21.15.33:6789/0 conn(0x7f614814f660 legacy=0x7f61480c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61477fe700 1 -- 172.21.15.33:0/826323109 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61480c21d0 con 0x7f614814f290 2023-10-06T02:45:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61477fe700 1 --2- 172.21.15.33:0/826323109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614814f290 0x7f61480c9f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6138009860 tx=0x7f61380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971e6ef641969d4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 -- 172.21.15.33:0/1716433268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 msgr2=0x7f87300c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 --2- 172.21.15.33:0/1716433268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f87300c65b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f873005e9b0 tx=0x7f872c009250 comp rx=0 tx=0).stop 2023-10-06T02:45:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61467fc700 1 -- 172.21.15.33:0/826323109 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138010590 con 0x7f614814f290 2023-10-06T02:45:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61467fc700 1 -- 172.21.15.33:0/826323109 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380106f0 con 0x7f614814f290 2023-10-06T02:45:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 -- 172.21.15.33:0/1716433268 shutdown_connections 2023-10-06T02:45:45.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 --2- 172.21.15.33:0/1716433268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f87300c65b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 --2- 172.21.15.33:0/1716433268 >> v2:172.21.15.167:3300/0 conn(0x7f8730154ab0 0x7f873006a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 -- 172.21.15.33:0/1716433268 >> 172.21.15.33:0/1716433268 conn(0x7f87300cc290 msgr2=0x7f87300c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f61467fc700 1 -- 172.21.15.33:0/826323109 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138010a90 con 0x7f614814f290 2023-10-06T02:45:45.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 -- 172.21.15.33:0/1716433268 shutdown_connections 2023-10-06T02:45:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f873736b700 1 -- 172.21.15.33:0/1716433268 wait complete. 2023-10-06T02:45:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- 172.21.15.33:0/826323109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614814f290 msgr2=0x7f61480c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 --2- 172.21.15.33:0/826323109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614814f290 0x7f61480c9f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6138009860 tx=0x7f61380091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- 172.21.15.33:0/826323109 shutdown_connections 2023-10-06T02:45:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 --2- 172.21.15.33:0/826323109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614814f290 0x7f61480c9f60 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6138009860 tx=0x7f61380091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- 172.21.15.33:0/826323109 >> 172.21.15.33:0/826323109 conn(0x7f61480c83f0 msgr2=0x7f61480c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.765+0000 7f614db50700 1 -- 172.21.15.33:0/826323109 shutdown_connections 2023-10-06T02:45:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 -- 172.21.15.33:0/826323109 wait complete. 2023-10-06T02:45:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 Processor -- start 2023-10-06T02:45:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 -- start start 2023-10-06T02:45:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 0x7f873014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8730154ab0 0x7f87300c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f873014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300c1ac0 con 0x7f8730154e80 2023-10-06T02:45:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873014ead0 con 0x7f87300694d0 2023-10-06T02:45:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873014d4f0 con 0x7f8730154ab0 2023-10-06T02:45:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 Processor -- start 2023-10-06T02:45:45.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736b6a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f873014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 0x7f873014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 0x7f873014cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43860/0 (socket says 172.21.15.33:43860) 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 -- 172.21.15.33:0/438465827 learned_addr learned my addr 172.21.15.33:0/438465827 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8735b68700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8730154ab0 0x7f87300c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 -- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8730154ab0 msgr2=0x7f87300c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8730154ab0 0x7f87300c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 -- 172.21.15.33:0/438465827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 msgr2=0x7f873014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f873014e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 -- 172.21.15.33:0/438465827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c0094f0 con 0x7f87300694d0 2023-10-06T02:45:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f8736369700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 0x7f873014cf00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f872800c470 tx=0x7f872800c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 -- start start 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 0x7f61480aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f614814f660 0x7f61480b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61481693a0 0x7f614814bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61480b1760 con 0x7f61480c1680 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148168f10 con 0x7f61481693a0 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f614db50700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61480af5e0 con 0x7f614814f660 2023-10-06T02:45:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 0x7f61480aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728007aa0 con 0x7f87300694d0 2023-10-06T02:45:45.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873014d7d0 con 0x7f87300694d0 2023-10-06T02:45:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 0x7f61480aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54844/0 (socket says 172.21.15.33:54844) 2023-10-06T02:45:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 -- 172.21.15.33:0/1077899183 learned_addr learned my addr 172.21.15.33:0/1077899183 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f6147fff700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61481693a0 0x7f614814bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f6146ffd700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f614814f660 0x7f61480b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728010040 con 0x7f87300694d0 2023-10-06T02:45:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728007e10 con 0x7f87300694d0 2023-10-06T02:45:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873014dd90 con 0x7f87300694d0 2023-10-06T02:45:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 -- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f614814f660 msgr2=0x7f61480b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f614814f660 0x7f61480b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 -- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61481693a0 msgr2=0x7f614814bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61481693a0 0x7f614814bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 -- 172.21.15.33:0/1077899183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6138009420 con 0x7f61480c1680 2023-10-06T02:45:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f6147fff700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61481693a0 0x7f614814bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.769+0000 7f61477fe700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 0x7f61480aef20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f614806ceb0 tx=0x7f6138017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f87280092e0 con 0x7f87300694d0 2023-10-06T02:45:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f87267fc700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8714060580 0x7f8714062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138010860 con 0x7f61480c1680 2023-10-06T02:45:45.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f872807d390 con 0x7f87300694d0 2023-10-06T02:45:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f8735b68700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8714060580 0x7f8714062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f614db50700 1 -- 172.21.15.33:0/1077899183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61480af8c0 con 0x7f61480c1680 2023-10-06T02:45:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f614db50700 1 -- 172.21.15.33:0/1077899183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6148168ba0 con 0x7f61480c1680 2023-10-06T02:45:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380109c0 con 0x7f61480c1680 2023-10-06T02:45:45.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613801d680 con 0x7f61480c1680 2023-10-06T02:45:45.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f8735b68700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8714060580 0x7f8714062a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f873005e9b0 tx=0x7f872c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 Processor -- start 2023-10-06T02:45:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 -- start start 2023-10-06T02:45:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5201696c0 0x7fe5200c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe5200683d0 0x7fe520169a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5200864b0 0x7fe520086930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200ca7e0 con 0x7fe5200683d0 2023-10-06T02:45:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200679c0 con 0x7fe5200864b0 2023-10-06T02:45:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe525990700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe520086e70 con 0x7fe5201696c0 2023-10-06T02:45:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51f7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5200864b0 0x7fe520086930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe5200683d0 0x7fe520169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51f7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe5200864b0 0x7fe520086930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43888/0 (socket says 172.21.15.33:43888) 2023-10-06T02:45:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51f7fe700 1 -- 172.21.15.33:0/1102746022 learned_addr learned my addr 172.21.15.33:0/1102746022 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51effd700 1 --2- 172.21.15.33:0/1102746022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5201696c0 0x7fe5200c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f613801d9f0 con 0x7f61480c1680 2023-10-06T02:45:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 -- 172.21.15.33:0/1102746022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5201696c0 msgr2=0x7fe5200c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 --2- 172.21.15.33:0/1102746022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5201696c0 0x7fe5200c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 -- 172.21.15.33:0/1102746022 >> v2:172.21.15.167:3300/0 conn(0x7fe5200864b0 msgr2=0x7fe520086930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 --2- 172.21.15.33:0/1102746022 >> v2:172.21.15.167:3300/0 conn(0x7fe5200864b0 0x7fe520086930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 -- 172.21.15.33:0/1102746022 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520067f40 con 0x7fe5200683d0 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51e7fc700 1 --2- 172.21.15.33:0/1102746022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520169a90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c7810 tx=0x7fe51800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf683d5a4b90da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51dffb700 1 -- 172.21.15.33:0/1102746022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518010070 con 0x7fe5200683d0 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51dffb700 1 -- 172.21.15.33:0/1102746022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51800da50 con 0x7fe5200683d0 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fe51dffb700 1 -- 172.21.15.33:0/1102746022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51800de10 con 0x7fe5200683d0 2023-10-06T02:45:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7f614db50700 1 -- 172.21.15.33:0/1077899183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61280052b0 con 0x7f61480c1680 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 Processor -- start 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 -- start start 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c1584a0 0x7f303c0ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f303c0caa20 0x7f303c0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303c06a180 con 0x7f303c0c9a80 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303c0c6480 con 0x7f303c0caa20 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304100d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c069570 con 0x7f303c1584a0 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f303c0caa20 0x7f303c0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f303c0caa20 0x7f303c0c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54858/0 (socket says 172.21.15.33:54858) 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 -- 172.21.15.33:0/2235880124 learned_addr learned my addr 172.21.15.33:0/2235880124 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f303bfff700 1 --2- 172.21.15.33:0/2235880124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c1584a0 0x7f303c0ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 -- 172.21.15.33:0/2235880124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c1584a0 msgr2=0x7f303c0ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 --2- 172.21.15.33:0/2235880124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c1584a0 0x7f303c0ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 -- 172.21.15.33:0/2235880124 >> v1:172.21.15.167:6789/0 conn(0x7f303c0c9a80 legacy=0x7f303c0ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 -- 172.21.15.33:0/2235880124 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c069850 con 0x7f303c0caa20 2023-10-06T02:45:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f304080c700 1 --2- 172.21.15.33:0/2235880124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c0caa20 0x7f303c0c7980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f302c0099b0 tx=0x7f302c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73839363b8a3662 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87180052b0 con 0x7f87300694d0 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- 172.21.15.33:0/1102746022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 msgr2=0x7fe520169a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- 172.21.15.33:0/1102746022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520169a90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c7810 tx=0x7fe51800b990 comp rx=0 tx=0).stop 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- 172.21.15.33:0/1102746022 shutdown_connections 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- 172.21.15.33:0/1102746022 >> v2:172.21.15.167:3300/0 conn(0x7fe5200864b0 0x7fe520086930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- 172.21.15.33:0/1102746022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520169a90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- 172.21.15.33:0/1102746022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5201696c0 0x7fe5200c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- 172.21.15.33:0/1102746022 >> 172.21.15.33:0/1102746022 conn(0x7fe5200c83f0 msgr2=0x7fe5200c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- 172.21.15.33:0/1102746022 shutdown_connections 2023-10-06T02:45:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- 172.21.15.33:0/1102746022 wait complete. 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 Processor -- start 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- start start 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520178a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5200864b0 0x7fe520069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5201696c0 0x7fe52006a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c47b0 con 0x7fe5200683d0 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c3ba0 con 0x7fe5201696c0 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe525990700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c3e80 con 0x7fe5200864b0 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520178a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520178a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54864/0 (socket says 172.21.15.33:54864) 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 -- 172.21.15.33:0/3643309054 learned_addr learned my addr 172.21.15.33:0/3643309054 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51e7fc700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5200864b0 0x7fe520069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51f7fe700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5201696c0 0x7fe52006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 -- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5200864b0 msgr2=0x7fe520069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5200864b0 0x7fe520069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 -- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5201696c0 msgr2=0x7fe52006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5201696c0 0x7fe52006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fe51effd700 1 -- 172.21.15.33:0/3643309054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51800bc30 con 0x7fe5200683d0 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fe51effd700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520178a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe514007fd0 tx=0x7fe51400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f612f7fe700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6134060550 0x7f6134062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f613807e8b0 con 0x7f61480c1680 2023-10-06T02:45:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f6146ffd700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6134060550 0x7f6134062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7f6146ffd700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6134060550 0x7f6134062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6148154ce0 tx=0x7f613c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514019070 con 0x7fe5200683d0 2023-10-06T02:45:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f303affd700 1 -- 172.21.15.33:0/2235880124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00b860 con 0x7f303c0caa20 2023-10-06T02:45:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5200c4160 con 0x7fe5200683d0 2023-10-06T02:45:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f613804e9f0 con 0x7f61480c1680 2023-10-06T02:45:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f303affd700 1 -- 172.21.15.33:0/2235880124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00b9c0 con 0x7f303c0caa20 2023-10-06T02:45:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f303affd700 1 -- 172.21.15.33:0/2235880124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00bd60 con 0x7f303c0caa20 2023-10-06T02:45:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe520178f40 con 0x7fe5200683d0 2023-10-06T02:45:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 Processor -- start 2023-10-06T02:45:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 -- start start 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb24c0c9550 0x7fb24c169a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0864b0 0x7fb24c086930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c067f40 con 0x7fb24c1696c0 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24c0ca7e0 con 0x7fb24c0c9550 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.773+0000 7fb254542700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0679c0 con 0x7fb24c0864b0 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fb2522de700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb24c1696c0 0x7fb24c0683d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33724/0 (socket says 172.21.15.33:33724) 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fb2522de700 1 -- 172.21.15.33:0/834523843 learned_addr learned my addr 172.21.15.33:0/834523843 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.777+0000 7fb252adf700 1 --2- 172.21.15.33:0/834523843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0864b0 0x7fb24c086930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb251add700 1 --2- 172.21.15.33:0/834523843 >> v2:172.21.15.167:3300/0 conn(0x7fb24c0c9550 0x7fb24c169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507408d700 1 Processor -- start 2023-10-06T02:45:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507408d700 1 -- start start 2023-10-06T02:45:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb252adf700 1 -- 172.21.15.33:0/834523843 >> v2:172.21.15.167:3300/0 conn(0x7fb24c0c9550 msgr2=0x7fb24c169a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb252adf700 1 --2- 172.21.15.33:0/834523843 >> v2:172.21.15.167:3300/0 conn(0x7fb24c0c9550 0x7fb24c169a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb252adf700 1 -- 172.21.15.33:0/834523843 >> v1:172.21.15.167:6789/0 conn(0x7fb24c1696c0 legacy=0x7fb24c0683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb252adf700 1 -- 172.21.15.33:0/834523843 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0c99d0 con 0x7fb24c0864b0 2023-10-06T02:45:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 -- 172.21.15.33:0/2235880124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c0caa20 msgr2=0x7f303c0c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 --2- 172.21.15.33:0/2235880124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c0caa20 0x7f303c0c7980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f302c0099b0 tx=0x7f302c009250 comp rx=0 tx=0).stop 2023-10-06T02:45:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb252adf700 1 --2- 172.21.15.33:0/834523843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0864b0 0x7fb24c086930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0ca920 tx=0x7fb23c009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d60acd2b663b6d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb2512dc700 1 -- 172.21.15.33:0/834523843 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00e070 con 0x7fb24c0864b0 2023-10-06T02:45:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb2512dc700 1 -- 172.21.15.33:0/834523843 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c0092e0 con 0x7fb24c0864b0 2023-10-06T02:45:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb2512dc700 1 -- 172.21.15.33:0/834523843 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00be60 con 0x7fb24c0864b0 2023-10-06T02:45:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507408d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c2510 0x7f506c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507408d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506c0c2080 con 0x7f506c0c0eb0 2023-10-06T02:45:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507408d700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506c0c1470 con 0x7f506c0c9e60 2023-10-06T02:45:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507408d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0c1750 con 0x7f506c0c2510 2023-10-06T02:45:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507262a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f506c0c0eb0 0x7f506c1587f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44794/0 (socket says 172.21.15.33:44794) 2023-10-06T02:45:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f507262a700 1 -- 172.21.15.33:0/2209270114 learned_addr learned my addr 172.21.15.33:0/2209270114 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f5071628700 1 --2- 172.21.15.33:0/2209270114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c2510 0x7f506c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb254542700 1 -- 172.21.15.33:0/834523843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0864b0 msgr2=0x7fb24c086930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb254542700 1 --2- 172.21.15.33:0/834523843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0864b0 0x7fb24c086930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0ca920 tx=0x7fb23c009f00 comp rx=0 tx=0).stop 2023-10-06T02:45:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 -- 172.21.15.33:0/2235880124 shutdown_connections 2023-10-06T02:45:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 --2- 172.21.15.33:0/2235880124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c0caa20 0x7f303c0c7980 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 --2- 172.21.15.33:0/2235880124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c1584a0 0x7f303c0ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 -- 172.21.15.33:0/2235880124 >> 172.21.15.33:0/2235880124 conn(0x7f303c0c8240 msgr2=0x7f303c0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 -- 172.21.15.33:0/2235880124 shutdown_connections 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f304100d700 1 -- 172.21.15.33:0/2235880124 wait complete. 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f5071628700 1 -- 172.21.15.33:0/2209270114 >> v1:172.21.15.167:6789/0 conn(0x7f506c0c9e60 legacy=0x7f506c068340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7f5071628700 1 -- 172.21.15.33:0/2209270114 >> v1:172.21.15.33:6789/0 conn(0x7f506c0c0eb0 legacy=0x7f506c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f5071628700 1 -- 172.21.15.33:0/2209270114 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c0c1a30 con 0x7f506c0c2510 2023-10-06T02:45:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb254542700 1 -- 172.21.15.33:0/834523843 shutdown_connections 2023-10-06T02:45:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb254542700 1 --2- 172.21.15.33:0/834523843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0864b0 0x7fb24c086930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb254542700 1 --2- 172.21.15.33:0/834523843 >> v2:172.21.15.167:3300/0 conn(0x7fb24c0c9550 0x7fb24c169a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fb254542700 1 -- 172.21.15.33:0/834523843 >> 172.21.15.33:0/834523843 conn(0x7fb24c0c83f0 msgr2=0x7fb24c0c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f5071628700 1 --2- 172.21.15.33:0/2209270114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c2510 0x7f506c0ca230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f505c009910 tx=0x7f505c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60177632c845ace8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdbc9bd700 1 Processor -- start 2023-10-06T02:45:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51400ac90 con 0x7fe5200683d0 2023-10-06T02:45:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.781+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514011040 con 0x7fe5200683d0 2023-10-06T02:45:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5000052b0 con 0x7fe5200683d0 2023-10-06T02:45:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdbc9bd700 1 -- start start 2023-10-06T02:45:45.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f5070e27700 1 -- 172.21.15.33:0/2209270114 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00b760 con 0x7f506c0c2510 2023-10-06T02:45:45.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f5070e27700 1 -- 172.21.15.33:0/2209270114 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c00b8c0 con 0x7f506c0c2510 2023-10-06T02:45:45.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f5070e27700 1 -- 172.21.15.33:0/2209270114 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00bc60 con 0x7f506c0c2510 2023-10-06T02:45:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fb254542700 1 -- 172.21.15.33:0/834523843 shutdown_connections 2023-10-06T02:45:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 -- 172.21.15.33:0/2209270114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c2510 msgr2=0x7f506c0ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 --2- 172.21.15.33:0/2209270114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c2510 0x7f506c0ca230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f505c009910 tx=0x7f505c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f304100d700 1 Processor -- start 2023-10-06T02:45:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f304100d700 1 -- start start 2023-10-06T02:45:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe5140158c0 con 0x7fe5200683d0 2023-10-06T02:45:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdbc9bd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdb8066fb0 0x7fbdb80c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdbc9bd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb8068a10 con 0x7fbdb806a0c0 2023-10-06T02:45:45.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdbc9bd700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb8068730 con 0x7fbdb80aeef0 2023-10-06T02:45:45.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdbc9bd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80ae240 con 0x7fbdb8066fb0 2023-10-06T02:45:45.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbdb806a0c0 0x7fbdb80ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44796/0 (socket says 172.21.15.33:44796) 2023-10-06T02:45:45.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb659c700 1 -- 172.21.15.33:0/741341496 learned_addr learned my addr 172.21.15.33:0/741341496 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:45.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb6d9d700 1 --2- 172.21.15.33:0/741341496 >> v2:172.21.15.33:3300/0 conn(0x7fbdb8066fb0 0x7fbdb80c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fb254542700 1 -- 172.21.15.33:0/834523843 wait complete. 2023-10-06T02:45:45.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/741341496 >> v1:172.21.15.167:6789/0 conn(0x7fbdb80aeef0 legacy=0x7fbdb8160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/741341496 >> v1:172.21.15.33:6789/0 conn(0x7fbdb806a0c0 legacy=0x7fbdb80ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 -- 172.21.15.33:0/2209270114 shutdown_connections 2023-10-06T02:45:45.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 --2- 172.21.15.33:0/2209270114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c2510 0x7f506c0ca230 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 -- 172.21.15.33:0/2209270114 >> 172.21.15.33:0/2209270114 conn(0x7f506c0c83f0 msgr2=0x7f506c0c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/741341496 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80cb3c0 con 0x7fbdb8066fb0 2023-10-06T02:45:45.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 -- 172.21.15.33:0/2209270114 shutdown_connections 2023-10-06T02:45:45.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fbdb6d9d700 1 --2- 172.21.15.33:0/741341496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb8066fb0 0x7fbdb80c8d20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e60 tx=0x7fbdb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad430e8165ddc5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fe506ffd700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe50c060550 0x7fe50c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7f507408d700 1 -- 172.21.15.33:0/2209270114 wait complete. 2023-10-06T02:45:45.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe51407d270 con 0x7fe5200683d0 2023-10-06T02:45:45.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fe51e7fc700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe50c060550 0x7fe50c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fe51e7fc700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe50c060550 0x7fe50c062a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b9d0 tx=0x7fe51801c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fb254542700 1 Processor -- start 2023-10-06T02:45:45.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.785+0000 7fb254542700 1 -- start start 2023-10-06T02:45:45.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 Processor -- start 2023-10-06T02:45:45.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 -- start start 2023-10-06T02:45:45.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304100d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 0x7f303c0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304100d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c0caa20 0x7f303c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304100d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 0x7f303c0c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304100d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c156830 con 0x7f303c1584a0 2023-10-06T02:45:45.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304100d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c155c20 con 0x7f303c0c9a80 2023-10-06T02:45:45.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304100d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c155f00 con 0x7f303c0caa20 2023-10-06T02:45:45.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304080c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 0x7f303c0c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 0x7f303c0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 0x7f303c0acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43908/0 (socket says 172.21.15.33:43908) 2023-10-06T02:45:45.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 -- 172.21.15.33:0/2090103319 learned_addr learned my addr 172.21.15.33:0/2090103319 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303b7fe700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c0caa20 0x7f303c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb254542700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 0x7fb24c0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb254542700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c0c9550 0x7fb24c176640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 0x7fb24c0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 0x7fb24c0ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54890/0 (socket says 172.21.15.33:54890) 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb254542700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0afe50 0x7fb24c0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 -- 172.21.15.33:0/133196524 learned_addr learned my addr 172.21.15.33:0/133196524 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0af9c0 con 0x7fb24c0864b0 2023-10-06T02:45:45.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c176b80 con 0x7fb24c0c9550 2023-10-06T02:45:45.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0c46b0 con 0x7fb24c0afe50 2023-10-06T02:45:45.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f872804d530 con 0x7f87300694d0 2023-10-06T02:45:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 -- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c0caa20 msgr2=0x7f303c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c0caa20 0x7f303c156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 -- 172.21.15.33:0/2090103319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 msgr2=0x7f303c0c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304080c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 0x7f303c0c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54886/0 (socket says 172.21.15.33:54886) 2023-10-06T02:45:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 0x7f303c0c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 -- 172.21.15.33:0/2090103319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0094f0 con 0x7f303c0c9a80 2023-10-06T02:45:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f304080c700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 0x7f303c0c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:45.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f303bfff700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 0x7f303c0acb40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f303000d860 tx=0x7f3030008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb251add700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c0c9550 0x7fb24c176640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb252adf700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0afe50 0x7fb24c0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 -- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0afe50 msgr2=0x7fb24c0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0afe50 0x7fb24c0c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 -- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c0c9550 msgr2=0x7fb24c176640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c0c9550 0x7fb24c176640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0071e0 con 0x7fb24c0864b0 2023-10-06T02:45:45.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb252adf700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0afe50 0x7fb24c0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:45.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c0eb0 0x7f506c14c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f506c0c2510 0x7f506c0c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 0x7f506c0c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c14c610 con 0x7f506c0c9e60 2023-10-06T02:45:45.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0c4e40 con 0x7f506c0c2510 2023-10-06T02:45:45.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507408d700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0aa130 con 0x7f506c0c0eb0 2023-10-06T02:45:45.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 0x7f506c0c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f5071628700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f506c0c2510 0x7f506c0c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f5071e29700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c0eb0 0x7f506c14c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 0x7f506c0c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54900/0 (socket says 172.21.15.33:54900) 2023-10-06T02:45:45.839 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 -- 172.21.15.33:0/1704279847 learned_addr learned my addr 172.21.15.33:0/1704279847 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7fb2522de700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 0x7fb24c0ae720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c590 tx=0x7fb248007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 -- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c0eb0 msgr2=0x7f506c14c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c0eb0 0x7f506c14c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 -- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f506c0c2510 msgr2=0x7f506c0c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f506c0c2510 0x7f506c0c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 -- 172.21.15.33:0/1704279847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c009420 con 0x7f506c0c9e60 2023-10-06T02:45:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.789+0000 7f507262a700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 0x7f506c0c2f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f506400cdd0 tx=0x7f5064001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdb5d9b700 1 -- 172.21.15.33:0/741341496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000e070 con 0x7fbdb8066fb0 2023-10-06T02:45:45.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdb5d9b700 1 -- 172.21.15.33:0/741341496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb000bcc0 con 0x7fbdb8066fb0 2023-10-06T02:45:45.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdb5d9b700 1 -- 172.21.15.33:0/741341496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0012680 con 0x7fbdb8066fb0 2023-10-06T02:45:45.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/741341496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb8066fb0 msgr2=0x7fbdb80c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/741341496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb8066fb0 0x7fbdb80c8d20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e60 tx=0x7fbdb0009780 comp rx=0 tx=0).stop 2023-10-06T02:45:45.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248005c00 con 0x7fb24c0864b0 2023-10-06T02:45:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb248005d80 con 0x7fb24c0864b0 2023-10-06T02:45:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248004a80 con 0x7fb24c0864b0 2023-10-06T02:45:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c0aedb0 con 0x7fb24c0864b0 2023-10-06T02:45:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c0af370 con 0x7fb24c0864b0 2023-10-06T02:45:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/741341496 shutdown_connections 2023-10-06T02:45:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/741341496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb8066fb0 0x7fbdb80c8d20 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e60 tx=0x7fbdb0009780 comp rx=0 tx=0).stop 2023-10-06T02:45:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/741341496 >> 172.21.15.33:0/741341496 conn(0x7fbdb80cc290 msgr2=0x7fbdb80c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/741341496 shutdown_connections 2023-10-06T02:45:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/741341496 wait complete. 2023-10-06T02:45:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2340052b0 con 0x7fb24c0864b0 2023-10-06T02:45:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.793+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb248005030 con 0x7fb24c0864b0 2023-10-06T02:45:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506400aa40 con 0x7f506c0c9e60 2023-10-06T02:45:45.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fb2427fc700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb230060550 0x7fb230062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb24807c540 con 0x7fb24c0864b0 2023-10-06T02:45:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506400aba0 con 0x7f506c0c9e60 2023-10-06T02:45:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c0a9520 con 0x7f506c0c9e60 2023-10-06T02:45:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c0a9ae0 con 0x7f506c0c9e60 2023-10-06T02:45:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fb251add700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb230060550 0x7fb230062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 Processor -- start 2023-10-06T02:45:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fb251add700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb230060550 0x7fb230062a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb24c086480 tx=0x7fb24400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50640070f0 con 0x7f506c0c9e60 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 -- start start 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 0x7fbdb81460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb80aeef0 0x7fbdb80b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb80b0af0 0x7fbdb8164c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb81547b0 con 0x7fbdb80aeef0 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b0120 con 0x7fbdb80b0af0 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdbc9bd700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80af550 con 0x7fbdb806a0c0 2023-10-06T02:45:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 0x7fbdb81460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 0x7fbdb81460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54814/0 (socket says 172.21.15.33:54814) 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/3023648404 learned_addr learned my addr 172.21.15.33:0/3023648404 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f50640074a0 con 0x7f506c0c9e60 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c0710b0 con 0x7f506c0c9e60 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5061ffb700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5050060650 0x7f5050062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f506407cf10 con 0x7f506c0c9e60 2023-10-06T02:45:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7f5071e29700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5050060650 0x7f5050062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb80b0af0 msgr2=0x7fbdb8164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb80b0af0 0x7fbdb8164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/3023648404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb80aeef0 msgr2=0x7fbdb80b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb80aeef0 0x7fbdb80b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 -- 172.21.15.33:0/3023648404 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0009a20 con 0x7fbdb806a0c0 2023-10-06T02:45:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.797+0000 7fbdb6d9d700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 0x7fbdb81460e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0013000 tx=0x7fbdb0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300125f0 con 0x7f303c0c9a80 2023-10-06T02:45:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030012750 con 0x7f303c0c9a80 2023-10-06T02:45:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c1561e0 con 0x7f303c0c9a80 2023-10-06T02:45:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030012af0 con 0x7f303c0c9a80 2023-10-06T02:45:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c1558b0 con 0x7f303c0c9a80 2023-10-06T02:45:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0027470 con 0x7fbdb806a0c0 2023-10-06T02:45:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7f5071e29700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5050060650 0x7f5050062b00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f506c06e8b0 tx=0x7f5068009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb00275f0 con 0x7fbdb806a0c0 2023-10-06T02:45:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb80af7f0 con 0x7fbdb806a0c0 2023-10-06T02:45:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe51404d410 con 0x7fe5200683d0 2023-10-06T02:45:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00279b0 con 0x7fbdb806a0c0 2023-10-06T02:45:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb80afdb0 con 0x7fbdb806a0c0 2023-10-06T02:45:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.801+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb8071040 con 0x7fbdb806a0c0 2023-10-06T02:45:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.805+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c0052b0 con 0x7f303c0c9a80 2023-10-06T02:45:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.805+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb24804c6e0 con 0x7fb24c0864b0 2023-10-06T02:45:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f303001e460 con 0x7f303c0c9a80 2023-10-06T02:45:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7f3023fff700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f30280606a0 0x7f3028062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f303007d110 con 0x7f303c0c9a80 2023-10-06T02:45:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbdb0024070 con 0x7fbdb806a0c0 2023-10-06T02:45:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7fbd9effd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbda4060550 0x7fbda4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fbdb007f7b0 con 0x7fbdb806a0c0 2023-10-06T02:45:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.809+0000 7fbdb659c700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbda4060550 0x7fbda4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.813+0000 7f303b7fe700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f30280606a0 0x7f3028062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.813+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbdb0081d50 con 0x7fbdb806a0c0 2023-10-06T02:45:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.813+0000 7f303b7fe700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f30280606a0 0x7f3028062b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f303c150210 tx=0x7f3034000f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.813+0000 7fbdb659c700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbda4060550 0x7fbda4062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e0e0 tx=0x7fbdac001350 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.813+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f506404d210 con 0x7f506c0c9e60 2023-10-06T02:45:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:45.813+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3030050d20 con 0x7f303c0c9a80 2023-10-06T02:45:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.065+0000 7f8b6a655700 1 -- 172.21.15.33:0/2404965150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b40005560 con 0x7f8b64154ab0 2023-10-06T02:45:46.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.065+0000 7f8b597fa700 1 -- 172.21.15.33:0/2404965150 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b5404f6b0 con 0x7f8b64154ab0 2023-10-06T02:45:46.074 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:46.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 -- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b480605d0 msgr2=0x7f8b48062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b480605d0 0x7f8b48062a80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c008160 tx=0x7f8b5c00d040 comp rx=0 tx=0).stop 2023-10-06T02:45:46.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 -- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b64154ab0 msgr2=0x7f8b64151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b64154ab0 0x7f8b64151200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c5c0 tx=0x7f8b54007e60 comp rx=0 tx=0).stop 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 -- 172.21.15.33:0/2404965150 shutdown_connections 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b480605d0 0x7f8b48062a80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b64154e80 0x7f8b641565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b64154ab0 0x7f8b64151200 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 --2- 172.21.15.33:0/2404965150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b640694d0 0x7f8b640c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 -- 172.21.15.33:0/2404965150 >> 172.21.15.33:0/2404965150 conn(0x7f8b640cc290 msgr2=0x7f8b640c1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 -- 172.21.15.33:0/2404965150 shutdown_connections 2023-10-06T02:45:46.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.073+0000 7f8b4effd700 1 -- 172.21.15.33:0/2404965150 wait complete. 2023-10-06T02:45:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.081+0000 7f614db50700 1 -- 172.21.15.33:0/1077899183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6128005560 con 0x7f61480c1680 2023-10-06T02:45:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.081+0000 7f612f7fe700 1 -- 172.21.15.33:0/1077899183 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6138052010 con 0x7f61480c1680 2023-10-06T02:45:46.090 INFO:teuthology.orchestra.run.smithi033.stdout:60129542153 2023-10-06T02:45:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 -- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6134060550 msgr2=0x7f6134062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6134060550 0x7f6134062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6148154ce0 tx=0x7f613c007ec0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 -- 172.21.15.33:0/1077899183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 msgr2=0x7f61480aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 0x7f61480aef20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f614806ceb0 tx=0x7f6138017000 comp rx=0 tx=0).stop 2023-10-06T02:45:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 -- 172.21.15.33:0/1077899183 shutdown_connections 2023-10-06T02:45:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6134060550 0x7f6134062a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f614814f660 0x7f61480b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61481693a0 0x7f614814bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 --2- 172.21.15.33:0/1077899183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480c1680 0x7f61480aef20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 -- 172.21.15.33:0/1077899183 >> 172.21.15.33:0/1077899183 conn(0x7f61480c83f0 msgr2=0x7f6148156600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.3 2023-10-06T02:45:46.093 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 -- 172.21.15.33:0/1077899183 shutdown_connections 2023-10-06T02:45:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.085+0000 7f612d7fa700 1 -- 172.21.15.33:0/1077899183 wait complete. 2023-10-06T02:45:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.093+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8718005830 con 0x7f87300694d0 2023-10-06T02:45:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.093+0000 7f87267fc700 1 -- 172.21.15.33:0/438465827 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8728050b50 con 0x7f87300694d0 2023-10-06T02:45:46.099 INFO:teuthology.orchestra.run.smithi033.stdout:60129542152 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8714060580 msgr2=0x7f8714062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8714060580 0x7f8714062a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f873005e9b0 tx=0x7f872c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 msgr2=0x7f873014cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 0x7f873014cf00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f872800c470 tx=0x7f872800c040 comp rx=0 tx=0).stop 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 shutdown_connections 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8714060580 0x7f8714062a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8730154ab0 0x7f87300c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8730154e80 0x7f873014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 --2- 172.21.15.33:0/438465827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87300694d0 0x7f873014cf00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 >> 172.21.15.33:0/438465827 conn(0x7f87300cc290 msgr2=0x7f87300ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.5 2023-10-06T02:45:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 shutdown_connections 2023-10-06T02:45:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.101+0000 7f873736b700 1 -- 172.21.15.33:0/438465827 wait complete. 2023-10-06T02:45:46.107 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.0 2023-10-06T02:45:46.115 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.113+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe500005830 con 0x7fe5200683d0 2023-10-06T02:45:46.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.113+0000 7fe506ffd700 1 -- 172.21.15.33:0/3643309054 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe514050a30 con 0x7fe5200683d0 2023-10-06T02:45:46.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.113+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f301c005560 con 0x7f303c0c9a80 2023-10-06T02:45:46.118 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:46.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.113+0000 7f3023fff700 1 -- 172.21.15.33:0/2090103319 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30300508b0 con 0x7f303c0c9a80 2023-10-06T02:45:46.118 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe50c060550 msgr2=0x7fe50c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe50c060550 0x7fe50c062a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe51800b9d0 tx=0x7fe51801c000 comp rx=0 tx=0).stop 2023-10-06T02:45:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 msgr2=0x7fe520178a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520178a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe514007fd0 tx=0x7fe51400e040 comp rx=0 tx=0).stop 2023-10-06T02:45:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f30280606a0 msgr2=0x7f3028062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f30280606a0 0x7f3028062b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f303c150210 tx=0x7f3034000f40 comp rx=0 tx=0).stop 2023-10-06T02:45:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 msgr2=0x7f303c0acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 0x7f303c0acb40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f303000d860 tx=0x7f3030008040 comp rx=0 tx=0).stop 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 shutdown_connections 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f30280606a0 0x7f3028062b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f303c0caa20 0x7f303c156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f303c1584a0 0x7f303c0c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 --2- 172.21.15.33:0/2090103319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f303c0c9a80 0x7f303c0acb40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 >> 172.21.15.33:0/2090103319 conn(0x7f303c0c8240 msgr2=0x7f303c1508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 shutdown_connections 2023-10-06T02:45:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe50c060550 0x7fe50c062a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5200864b0 0x7fe520069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5201696c0 0x7fe52006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 --2- 172.21.15.33:0/3643309054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5200683d0 0x7fe520178a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 >> 172.21.15.33:0/3643309054 conn(0x7fe5200c83f0 msgr2=0x7fe5200c2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 shutdown_connections 2023-10-06T02:45:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 shutdown_connections 2023-10-06T02:45:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7fe525990700 1 -- 172.21.15.33:0/3643309054 wait complete. 2023-10-06T02:45:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.117+0000 7f304100d700 1 -- 172.21.15.33:0/2090103319 wait complete. 2023-10-06T02:45:46.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.121+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f506c0c1d10 con 0x7f506c0c9e60 2023-10-06T02:45:46.128 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.121+0000 7f5061ffb700 1 -- 172.21.15.33:0/1704279847 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5064050830 con 0x7f506c0c9e60 2023-10-06T02:45:46.128 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5050060650 msgr2=0x7f5050062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5050060650 0x7f5050062b00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f506c06e8b0 tx=0x7f5068009040 comp rx=0 tx=0).stop 2023-10-06T02:45:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 msgr2=0x7f506c0c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 0x7f506c0c2f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f506400cdd0 tx=0x7f5064001b90 comp rx=0 tx=0).stop 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 shutdown_connections 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c0c9e60 0x7f506c0c2f70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f506c0c2510 0x7f506c0c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5050060650 0x7f5050062b00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 --2- 172.21.15.33:0/1704279847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f506c0c0eb0 0x7f506c14c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 >> 172.21.15.33:0/1704279847 conn(0x7f506c0c83f0 msgr2=0x7f506c15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 shutdown_connections 2023-10-06T02:45:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.125+0000 7f507408d700 1 -- 172.21.15.33:0/1704279847 wait complete. 2023-10-06T02:45:46.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.7 2023-10-06T02:45:46.133 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.2 2023-10-06T02:45:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.129+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb234005830 con 0x7fb24c0864b0 2023-10-06T02:45:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.129+0000 7fb2427fc700 1 -- 172.21.15.33:0/133196524 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb24804fd00 con 0x7fb24c0864b0 2023-10-06T02:45:46.135 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.135 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.129+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbdb80c8e80 con 0x7fbdb806a0c0 2023-10-06T02:45:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.129+0000 7fbd9effd700 1 -- 172.21.15.33:0/3023648404 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbdb004f8b0 con 0x7fbdb806a0c0 2023-10-06T02:45:46.139 INFO:teuthology.orchestra.run.smithi033.stdout:60129542154 2023-10-06T02:45:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb230060550 msgr2=0x7fb230062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb230060550 0x7fb230062a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb24c086480 tx=0x7fb24400b040 comp rx=0 tx=0).stop 2023-10-06T02:45:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 msgr2=0x7fb24c0ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 0x7fb24c0ae720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c590 tx=0x7fb248007ee0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 shutdown_connections 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb230060550 0x7fb230062a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb24c0afe50 0x7fb24c0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c0c9550 0x7fb24c176640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 --2- 172.21.15.33:0/133196524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb24c0864b0 0x7fb24c0ae720 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 >> 172.21.15.33:0/133196524 conn(0x7fb24c0c83f0 msgr2=0x7fb24c1654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 shutdown_connections 2023-10-06T02:45:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fb254542700 1 -- 172.21.15.33:0/133196524 wait complete. 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbda4060550 msgr2=0x7fbda4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbda4060550 0x7fbda4062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e0e0 tx=0x7fbdac001350 comp rx=0 tx=0).stop 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 msgr2=0x7fbdb81460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 0x7fbdb81460e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0013000 tx=0x7fbdb0019000 comp rx=0 tx=0).stop 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 shutdown_connections 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdb80b0af0 0x7fbdb8164c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdb80aeef0 0x7fbdb80b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbda4060550 0x7fbda4062a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 --2- 172.21.15.33:0/3023648404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdb806a0c0 0x7fbdb81460e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 >> 172.21.15.33:0/3023648404 conn(0x7fbdb80cc290 msgr2=0x7fbdb80c0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 shutdown_connections 2023-10-06T02:45:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.133+0000 7fbdbc9bd700 1 -- 172.21.15.33:0/3023648404 wait complete. 2023-10-06T02:45:46.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.4 2023-10-06T02:45:46.145 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.6 2023-10-06T02:45:46.149 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.1 2023-10-06T02:45:46.150 DEBUG:teuthology.parallel:result is None 2023-10-06T02:45:46.150 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T02:45:46.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:45:46.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.289+0000 7f5680827700 1 Processor -- start 2023-10-06T02:45:46.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.289+0000 7f5680827700 1 -- start start 2023-10-06T02:45:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f567c0adb60 0x7f567c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c069820 con 0x7f567c0adf30 2023-10-06T02:45:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567c067ea0 con 0x7f567c0adb60 2023-10-06T02:45:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c068180 con 0x7f567c067970 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f567c0adf30 0x7f567c156ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44812/0 (socket says 172.21.15.33:44812) 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f567c0adb60 0x7f567c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567bfff700 1 -- 172.21.15.33:0/2795364285 learned_addr learned my addr 172.21.15.33:0/2795364285 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567affd700 1 --2- 172.21.15.33:0/2795364285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567a7fc700 1 -- 172.21.15.33:0/2795364285 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413250138 0 0) 0x7f567c069820 con 0x7f567c0adf30 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567b7fe700 1 -- 172.21.15.33:0/2795364285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 msgr2=0x7f567c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567b7fe700 1 --2- 172.21.15.33:0/2795364285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567b7fe700 1 -- 172.21.15.33:0/2795364285 >> v1:172.21.15.33:6789/0 conn(0x7f567c0adf30 legacy=0x7f567c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567b7fe700 1 -- 172.21.15.33:0/2795364285 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c068460 con 0x7f567c0adb60 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567b7fe700 1 --2- 172.21.15.33:0/2795364285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adb60 0x7f567c0c6820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f566c0098e0 tx=0x7f566c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb33fcf7977f136 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567a7fc700 1 -- 172.21.15.33:0/2795364285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00b640 con 0x7f567c0adb60 2023-10-06T02:45:46.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567a7fc700 1 -- 172.21.15.33:0/2795364285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c00b7a0 con 0x7f567c0adb60 2023-10-06T02:45:46.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f567a7fc700 1 -- 172.21.15.33:0/2795364285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00bb40 con 0x7f567c0adb60 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- 172.21.15.33:0/2795364285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adb60 msgr2=0x7f567c0c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 --2- 172.21.15.33:0/2795364285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adb60 0x7f567c0c6820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f566c0098e0 tx=0x7f566c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- 172.21.15.33:0/2795364285 shutdown_connections 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 --2- 172.21.15.33:0/2795364285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 --2- 172.21.15.33:0/2795364285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adb60 0x7f567c0c6820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- 172.21.15.33:0/2795364285 >> 172.21.15.33:0/2795364285 conn(0x7f567c0c8240 msgr2=0x7f567c0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- 172.21.15.33:0/2795364285 shutdown_connections 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.293+0000 7f5680827700 1 -- 172.21.15.33:0/2795364285 wait complete. 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 Processor -- start 2023-10-06T02:45:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 -- start start 2023-10-06T02:45:46.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c0adb60 0x7f567c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 0x7f567c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c0c2b20 con 0x7f567c0adf30 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c0c93d0 con 0x7f567c0adb60 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c0ca230 con 0x7f567c067970 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 0x7f567c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 0x7f567c0ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54924/0 (socket says 172.21.15.33:54924) 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c0adb60 0x7f567c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 -- 172.21.15.33:0/4194245424 learned_addr learned my addr 172.21.15.33:0/4194245424 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567b7fe700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 -- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 msgr2=0x7f567c155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 -- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c0adb60 msgr2=0x7f567c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c0adb60 0x7f567c14a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 -- 172.21.15.33:0/4194245424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c009420 con 0x7f567c0adf30 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567b7fe700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f567bfff700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 0x7f567c0ca6c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f567400c530 tx=0x7f5674007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674005b30 con 0x7f567c0adf30 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674005c90 con 0x7f567c0adf30 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674007320 con 0x7f567c0adf30 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c14a6d0 con 0x7f567c0adf30 2023-10-06T02:45:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.297+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c0c9900 con 0x7f567c0adf30 2023-10-06T02:45:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.301+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c071040 con 0x7f567c0adf30 2023-10-06T02:45:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.301+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f567401b3b0 con 0x7f567c0adf30 2023-10-06T02:45:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.301+0000 7f56637fe700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5668060580 0x7f5668062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.301+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f567407c070 con 0x7f567c0adf30 2023-10-06T02:45:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.301+0000 7f567b7fe700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5668060580 0x7f5668062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.301+0000 7f567b7fe700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5668060580 0x7f5668062a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f566c0091e0 tx=0x7f566c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.305+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f567404c1f0 con 0x7f567c0adf30 2023-10-06T02:45:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.457+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f567c0ca3a0 con 0x7f5668060580 2023-10-06T02:45:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.477+0000 7f56637fe700 1 -- 172.21.15.33:0/4194245424 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367223 (secure 0 0 0) 0x7f567c0ca3a0 con 0x7f5668060580 2023-10-06T02:45:46.488 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5668060580 msgr2=0x7f5668062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5668060580 0x7f5668062a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f566c0091e0 tx=0x7f566c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 msgr2=0x7f567c0ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 0x7f567c0ca6c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f567400c530 tx=0x7f5674007f60 comp rx=0 tx=0).stop 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 shutdown_connections 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567c0adf30 0x7f567c0ca6c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c0adb60 0x7f567c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5668060580 0x7f5668062a30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 --2- 172.21.15.33:0/4194245424 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f567c067970 0x7f567c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 >> 172.21.15.33:0/4194245424 conn(0x7f567c0c8240 msgr2=0x7f567c0ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 shutdown_connections 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.485+0000 7f5680827700 1 -- 172.21.15.33:0/4194245424 wait complete. 2023-10-06T02:45:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:45:46.503 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-06T02:45:44.566000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381960,"kb_used_data":3656,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592760,"statfs":{"total":773094113280,"available":770654986240,"internally_reserved":0,"allocated":3743744,"data_stored":2209232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004937"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.352797+0000","last_change":"2023-10-06T02:45:27.343811+0000","last_active":"2023-10-06T02:45:27.352797+0000","last_peered":"2023-10-06T02:45:27.352797+0000","last_clean":"2023-10-06T02:45:27.352797+0000","last_became_active":"2023-10-06T02:45:27.343175+0000","last_became_peered":"2023-10-06T02:45:27.343175+0000","last_unstale":"2023-10-06T02:45:27.352797+0000","last_undegraded":"2023-10-06T02:45:27.352797+0000","last_fullsized":"2023-10-06T02:45:27.352797+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334994+0000","last_change":"2023-10-06T02:45:22.801080+0000","last_active":"2023-10-06T02:45:27.334994+0000","last_peered":"2023-10-06T02:45:27.334994+0000","last_clean":"2023-10-06T02:45:27.334994+0000","last_became_active":"2023-10-06T02:45:22.800763+0000","last_became_peered":"2023-10-06T02:45:22.800763+0000","last_unstale":"2023-10-06T02:45:27.334994+0000","last_undegraded":"2023-10-06T02:45:27.334994+0000","last_fullsized":"2023-10-06T02:45:27.334994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333783+0000","last_change":"2023-10-06T02:45:23.305500+0000","last_active":"2023-10-06T02:45:27.333783+0000","last_peered":"2023-10-06T02:45:27.333783+0000","last_clean":"2023-10-06T02:45:27.333783+0000","last_became_active":"2023-10-06T02:45:23.305199+0000","last_became_peered":"2023-10-06T02:45:23.305199+0000","last_unstale":"2023-10-06T02:45:27.333783+0000","last_undegraded":"2023-10-06T02:45:27.333783+0000","last_fullsized":"2023-10-06T02:45:27.333783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351635+0000","last_change":"2023-10-06T02:45:27.345953+0000","last_active":"2023-10-06T02:45:27.351635+0000","last_peered":"2023-10-06T02:45:27.351635+0000","last_clean":"2023-10-06T02:45:27.351635+0000","last_became_active":"2023-10-06T02:45:27.345526+0000","last_became_peered":"2023-10-06T02:45:27.345526+0000","last_unstale":"2023-10-06T02:45:27.351635+0000","last_undegraded":"2023-10-06T02:45:27.351635+0000","last_fullsized":"2023-10-06T02:45:27.351635+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:37.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332907+0000","last_change":"2023-10-06T02:45:22.805277+0000","last_active":"2023-10-06T02:45:27.332907+0000","last_peered":"2023-10-06T02:45:27.332907+0000","last_clean":"2023-10-06T02:45:27.332907+0000","last_became_active":"2023-10-06T02:45:22.804802+0000","last_became_peered":"2023-10-06T02:45:22.804802+0000","last_unstale":"2023-10-06T02:45:27.332907+0000","last_undegraded":"2023-10-06T02:45:27.332907+0000","last_fullsized":"2023-10-06T02:45:27.332907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:37.831567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330139+0000","last_change":"2023-10-06T02:45:23.306937+0000","last_active":"2023-10-06T02:45:27.330139+0000","last_peered":"2023-10-06T02:45:27.330139+0000","last_clean":"2023-10-06T02:45:27.330139+0000","last_became_active":"2023-10-06T02:45:23.306641+0000","last_became_peered":"2023-10-06T02:45:23.306641+0000","last_unstale":"2023-10-06T02:45:27.330139+0000","last_undegraded":"2023-10-06T02:45:27.330139+0000","last_fullsized":"2023-10-06T02:45:27.330139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:23.162123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356992+0000","last_change":"2023-10-06T02:45:27.353931+0000","last_active":"2023-10-06T02:45:27.356992+0000","last_peered":"2023-10-06T02:45:27.356992+0000","last_clean":"2023-10-06T02:45:27.356992+0000","last_became_active":"2023-10-06T02:45:27.353488+0000","last_became_peered":"2023-10-06T02:45:27.353488+0000","last_unstale":"2023-10-06T02:45:27.356992+0000","last_undegraded":"2023-10-06T02:45:27.356992+0000","last_fullsized":"2023-10-06T02:45:27.356992+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341505+0000","last_change":"2023-10-06T02:45:22.803357+0000","last_active":"2023-10-06T02:45:27.341505+0000","last_peered":"2023-10-06T02:45:27.341505+0000","last_clean":"2023-10-06T02:45:27.341505+0000","last_became_active":"2023-10-06T02:45:22.800673+0000","last_became_peered":"2023-10-06T02:45:22.800673+0000","last_unstale":"2023-10-06T02:45:27.341505+0000","last_undegraded":"2023-10-06T02:45:27.341505+0000","last_fullsized":"2023-10-06T02:45:27.341505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336470+0000","last_change":"2023-10-06T02:45:23.310269+0000","last_active":"2023-10-06T02:45:27.336470+0000","last_peered":"2023-10-06T02:45:27.336470+0000","last_clean":"2023-10-06T02:45:27.336470+0000","last_became_active":"2023-10-06T02:45:23.309927+0000","last_became_peered":"2023-10-06T02:45:23.309927+0000","last_unstale":"2023-10-06T02:45:27.336470+0000","last_undegraded":"2023-10-06T02:45:27.336470+0000","last_fullsized":"2023-10-06T02:45:27.336470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351290+0000","last_change":"2023-10-06T02:45:27.348258+0000","last_active":"2023-10-06T02:45:27.351290+0000","last_peered":"2023-10-06T02:45:27.351290+0000","last_clean":"2023-10-06T02:45:27.351290+0000","last_became_active":"2023-10-06T02:45:27.347848+0000","last_became_peered":"2023-10-06T02:45:27.347848+0000","last_unstale":"2023-10-06T02:45:27.351290+0000","last_undegraded":"2023-10-06T02:45:27.351290+0000","last_fullsized":"2023-10-06T02:45:27.351290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340794+0000","last_change":"2023-10-06T02:45:22.800086+0000","last_active":"2023-10-06T02:45:27.340794+0000","last_peered":"2023-10-06T02:45:27.340794+0000","last_clean":"2023-10-06T02:45:27.340794+0000","last_became_active":"2023-10-06T02:45:22.799739+0000","last_became_peered":"2023-10-06T02:45:22.799739+0000","last_unstale":"2023-10-06T02:45:27.340794+0000","last_undegraded":"2023-10-06T02:45:27.340794+0000","last_fullsized":"2023-10-06T02:45:27.340794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332497+0000","last_change":"2023-10-06T02:45:23.312373+0000","last_active":"2023-10-06T02:45:27.332497+0000","last_peered":"2023-10-06T02:45:27.332497+0000","last_clean":"2023-10-06T02:45:27.332497+0000","last_became_active":"2023-10-06T02:45:23.311513+0000","last_became_peered":"2023-10-06T02:45:23.311513+0000","last_unstale":"2023-10-06T02:45:27.332497+0000","last_undegraded":"2023-10-06T02:45:27.332497+0000","last_fullsized":"2023-10-06T02:45:27.332497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:10.715925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360984+0000","last_change":"2023-10-06T02:45:27.347884+0000","last_active":"2023-10-06T02:45:27.360984+0000","last_peered":"2023-10-06T02:45:27.360984+0000","last_clean":"2023-10-06T02:45:27.360984+0000","last_became_active":"2023-10-06T02:45:27.347285+0000","last_became_peered":"2023-10-06T02:45:27.347285+0000","last_unstale":"2023-10-06T02:45:27.360984+0000","last_undegraded":"2023-10-06T02:45:27.360984+0000","last_fullsized":"2023-10-06T02:45:27.360984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:52.629371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335221+0000","last_change":"2023-10-06T02:45:22.804322+0000","last_active":"2023-10-06T02:45:27.335221+0000","last_peered":"2023-10-06T02:45:27.335221+0000","last_clean":"2023-10-06T02:45:27.335221+0000","last_became_active":"2023-10-06T02:45:22.802485+0000","last_became_peered":"2023-10-06T02:45:22.802485+0000","last_unstale":"2023-10-06T02:45:27.335221+0000","last_undegraded":"2023-10-06T02:45:27.335221+0000","last_fullsized":"2023-10-06T02:45:27.335221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320001+0000","last_change":"2023-10-06T02:45:23.311929+0000","last_active":"2023-10-06T02:45:27.320001+0000","last_peered":"2023-10-06T02:45:27.320001+0000","last_clean":"2023-10-06T02:45:27.320001+0000","last_became_active":"2023-10-06T02:45:23.311209+0000","last_became_peered":"2023-10-06T02:45:23.311209+0000","last_unstale":"2023-10-06T02:45:27.320001+0000","last_undegraded":"2023-10-06T02:45:27.320001+0000","last_fullsized":"2023-10-06T02:45:27.320001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:39.089521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354213+0000","last_change":"2023-10-06T02:45:27.347958+0000","last_active":"2023-10-06T02:45:27.354213+0000","last_peered":"2023-10-06T02:45:27.354213+0000","last_clean":"2023-10-06T02:45:27.354213+0000","last_became_active":"2023-10-06T02:45:27.347490+0000","last_became_peered":"2023-10-06T02:45:27.347490+0000","last_unstale":"2023-10-06T02:45:27.354213+0000","last_undegraded":"2023-10-06T02:45:27.354213+0000","last_fullsized":"2023-10-06T02:45:27.354213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:40.393467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336567+0000","last_change":"2023-10-06T02:45:22.799339+0000","last_active":"2023-10-06T02:45:27.336567+0000","last_peered":"2023-10-06T02:45:27.336567+0000","last_clean":"2023-10-06T02:45:27.336567+0000","last_became_active":"2023-10-06T02:45:22.798982+0000","last_became_peered":"2023-10-06T02:45:22.798982+0000","last_unstale":"2023-10-06T02:45:27.336567+0000","last_undegraded":"2023-10-06T02:45:27.336567+0000","last_fullsized":"2023-10-06T02:45:27.336567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:41.206862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330449+0000","last_change":"2023-10-06T02:45:23.312269+0000","last_active":"2023-10-06T02:45:27.330449+0000","last_peered":"2023-10-06T02:45:27.330449+0000","last_clean":"2023-10-06T02:45:27.330449+0000","last_became_active":"2023-10-06T02:45:23.311975+0000","last_became_peered":"2023-10-06T02:45:23.311975+0000","last_unstale":"2023-10-06T02:45:27.330449+0000","last_undegraded":"2023-10-06T02:45:27.330449+0000","last_fullsized":"2023-10-06T02:45:27.330449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:27.851854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354212+0000","last_change":"2023-10-06T02:45:27.347553+0000","last_active":"2023-10-06T02:45:27.354212+0000","last_peered":"2023-10-06T02:45:27.354212+0000","last_clean":"2023-10-06T02:45:27.354212+0000","last_became_active":"2023-10-06T02:45:27.347165+0000","last_became_peered":"2023-10-06T02:45:27.347165+0000","last_unstale":"2023-10-06T02:45:27.354212+0000","last_undegraded":"2023-10-06T02:45:27.354212+0000","last_fullsized":"2023-10-06T02:45:27.354212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:38.681094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:22.808457+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:22.808145+0000","last_became_peered":"2023-10-06T02:45:22.808145+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334405+0000","last_change":"2023-10-06T02:45:23.323552+0000","last_active":"2023-10-06T02:45:27.334405+0000","last_peered":"2023-10-06T02:45:27.334405+0000","last_clean":"2023-10-06T02:45:27.334405+0000","last_became_active":"2023-10-06T02:45:23.323241+0000","last_became_peered":"2023-10-06T02:45:23.323241+0000","last_unstale":"2023-10-06T02:45:27.334405+0000","last_undegraded":"2023-10-06T02:45:27.334405+0000","last_fullsized":"2023-10-06T02:45:27.334405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353329+0000","last_change":"2023-10-06T02:45:27.349343+0000","last_active":"2023-10-06T02:45:27.353329+0000","last_peered":"2023-10-06T02:45:27.353329+0000","last_clean":"2023-10-06T02:45:27.353329+0000","last_became_active":"2023-10-06T02:45:27.348859+0000","last_became_peered":"2023-10-06T02:45:27.348859+0000","last_unstale":"2023-10-06T02:45:27.353329+0000","last_undegraded":"2023-10-06T02:45:27.353329+0000","last_fullsized":"2023-10-06T02:45:27.353329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333023+0000","last_change":"2023-10-06T02:45:22.811866+0000","last_active":"2023-10-06T02:45:27.333023+0000","last_peered":"2023-10-06T02:45:27.333023+0000","last_clean":"2023-10-06T02:45:27.333023+0000","last_became_active":"2023-10-06T02:45:22.811334+0000","last_became_peered":"2023-10-06T02:45:22.811334+0000","last_unstale":"2023-10-06T02:45:27.333023+0000","last_undegraded":"2023-10-06T02:45:27.333023+0000","last_fullsized":"2023-10-06T02:45:27.333023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325429+0000","last_change":"2023-10-06T02:45:23.320977+0000","last_active":"2023-10-06T02:45:27.325429+0000","last_peered":"2023-10-06T02:45:27.325429+0000","last_clean":"2023-10-06T02:45:27.325429+0000","last_became_active":"2023-10-06T02:45:23.320677+0000","last_became_peered":"2023-10-06T02:45:23.320677+0000","last_unstale":"2023-10-06T02:45:27.325429+0000","last_undegraded":"2023-10-06T02:45:27.325429+0000","last_fullsized":"2023-10-06T02:45:27.325429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:18.520846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332892+0000","last_change":"2023-10-06T02:45:23.312712+0000","last_active":"2023-10-06T02:45:27.332892+0000","last_peered":"2023-10-06T02:45:27.332892+0000","last_clean":"2023-10-06T02:45:27.332892+0000","last_became_active":"2023-10-06T02:45:23.312164+0000","last_became_peered":"2023-10-06T02:45:23.312164+0000","last_unstale":"2023-10-06T02:45:27.332892+0000","last_undegraded":"2023-10-06T02:45:27.332892+0000","last_fullsized":"2023-10-06T02:45:27.332892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.186960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335089+0000","last_change":"2023-10-06T02:45:22.800781+0000","last_active":"2023-10-06T02:45:27.335089+0000","last_peered":"2023-10-06T02:45:27.335089+0000","last_clean":"2023-10-06T02:45:27.335089+0000","last_became_active":"2023-10-06T02:45:22.800484+0000","last_became_peered":"2023-10-06T02:45:22.800484+0000","last_unstale":"2023-10-06T02:45:27.335089+0000","last_undegraded":"2023-10-06T02:45:27.335089+0000","last_fullsized":"2023-10-06T02:45:27.335089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354176+0000","last_change":"2023-10-06T02:45:27.348699+0000","last_active":"2023-10-06T02:45:27.354176+0000","last_peered":"2023-10-06T02:45:27.354176+0000","last_clean":"2023-10-06T02:45:27.354176+0000","last_became_active":"2023-10-06T02:45:27.348111+0000","last_became_peered":"2023-10-06T02:45:27.348111+0000","last_unstale":"2023-10-06T02:45:27.354176+0000","last_undegraded":"2023-10-06T02:45:27.354176+0000","last_fullsized":"2023-10-06T02:45:27.354176+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:26.404420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330041+0000","last_change":"2023-10-06T02:45:23.323471+0000","last_active":"2023-10-06T02:45:27.330041+0000","last_peered":"2023-10-06T02:45:27.330041+0000","last_clean":"2023-10-06T02:45:27.330041+0000","last_became_active":"2023-10-06T02:45:23.323203+0000","last_became_peered":"2023-10-06T02:45:23.323203+0000","last_unstale":"2023-10-06T02:45:27.330041+0000","last_undegraded":"2023-10-06T02:45:27.330041+0000","last_fullsized":"2023-10-06T02:45:27.330041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:59.446720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341624+0000","last_change":"2023-10-06T02:45:22.804313+0000","last_active":"2023-10-06T02:45:27.341624+0000","last_peered":"2023-10-06T02:45:27.341624+0000","last_clean":"2023-10-06T02:45:27.341624+0000","last_became_active":"2023-10-06T02:45:22.804008+0000","last_became_peered":"2023-10-06T02:45:22.804008+0000","last_unstale":"2023-10-06T02:45:27.341624+0000","last_undegraded":"2023-10-06T02:45:27.341624+0000","last_fullsized":"2023-10-06T02:45:27.341624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356829+0000","last_change":"2023-10-06T02:45:27.355277+0000","last_active":"2023-10-06T02:45:27.356829+0000","last_peered":"2023-10-06T02:45:27.356829+0000","last_clean":"2023-10-06T02:45:27.356829+0000","last_became_active":"2023-10-06T02:45:27.354885+0000","last_became_peered":"2023-10-06T02:45:27.354885+0000","last_unstale":"2023-10-06T02:45:27.356829+0000","last_undegraded":"2023-10-06T02:45:27.356829+0000","last_fullsized":"2023-10-06T02:45:27.356829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.722369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333001+0000","last_change":"2023-10-06T02:45:22.810372+0000","last_active":"2023-10-06T02:45:27.333001+0000","last_peered":"2023-10-06T02:45:27.333001+0000","last_clean":"2023-10-06T02:45:27.333001+0000","last_became_active":"2023-10-06T02:45:22.810117+0000","last_became_peered":"2023-10-06T02:45:22.810117+0000","last_unstale":"2023-10-06T02:45:27.333001+0000","last_undegraded":"2023-10-06T02:45:27.333001+0000","last_fullsized":"2023-10-06T02:45:27.333001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:54.333503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335943+0000","last_change":"2023-10-06T02:45:23.319583+0000","last_active":"2023-10-06T02:45:27.335943+0000","last_peered":"2023-10-06T02:45:27.335943+0000","last_clean":"2023-10-06T02:45:27.335943+0000","last_became_active":"2023-10-06T02:45:23.319266+0000","last_became_peered":"2023-10-06T02:45:23.319266+0000","last_unstale":"2023-10-06T02:45:27.335943+0000","last_undegraded":"2023-10-06T02:45:27.335943+0000","last_fullsized":"2023-10-06T02:45:27.335943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353966+0000","last_change":"2023-10-06T02:45:27.351069+0000","last_active":"2023-10-06T02:45:27.353966+0000","last_peered":"2023-10-06T02:45:27.353966+0000","last_clean":"2023-10-06T02:45:27.353966+0000","last_became_active":"2023-10-06T02:45:27.350579+0000","last_became_peered":"2023-10-06T02:45:27.350579+0000","last_unstale":"2023-10-06T02:45:27.353966+0000","last_undegraded":"2023-10-06T02:45:27.353966+0000","last_fullsized":"2023-10-06T02:45:27.353966+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:47.948820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"29'9","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:33.113646+0000","last_change":"2023-10-06T02:45:22.804024+0000","last_active":"2023-10-06T02:45:33.113646+0000","last_peered":"2023-10-06T02:45:33.113646+0000","last_clean":"2023-10-06T02:45:33.113646+0000","last_became_active":"2023-10-06T02:45:22.803679+0000","last_became_peered":"2023-10-06T02:45:22.803679+0000","last_unstale":"2023-10-06T02:45:33.113646+0000","last_undegraded":"2023-10-06T02:45:33.113646+0000","last_fullsized":"2023-10-06T02:45:33.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.507771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332592+0000","last_change":"2023-10-06T02:45:23.312638+0000","last_active":"2023-10-06T02:45:27.332592+0000","last_peered":"2023-10-06T02:45:27.332592+0000","last_clean":"2023-10-06T02:45:27.332592+0000","last_became_active":"2023-10-06T02:45:23.312006+0000","last_became_peered":"2023-10-06T02:45:23.312006+0000","last_unstale":"2023-10-06T02:45:27.332592+0000","last_undegraded":"2023-10-06T02:45:27.332592+0000","last_fullsized":"2023-10-06T02:45:27.332592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.358568+0000","last_change":"2023-10-06T02:45:27.345474+0000","last_active":"2023-10-06T02:45:27.358568+0000","last_peered":"2023-10-06T02:45:27.358568+0000","last_clean":"2023-10-06T02:45:27.358568+0000","last_became_active":"2023-10-06T02:45:27.345014+0000","last_became_peered":"2023-10-06T02:45:27.345014+0000","last_unstale":"2023-10-06T02:45:27.358568+0000","last_undegraded":"2023-10-06T02:45:27.358568+0000","last_fullsized":"2023-10-06T02:45:27.358568+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334989+0000","last_change":"2023-10-06T02:45:23.319302+0000","last_active":"2023-10-06T02:45:27.334989+0000","last_peered":"2023-10-06T02:45:27.334989+0000","last_clean":"2023-10-06T02:45:27.334989+0000","last_became_active":"2023-10-06T02:45:23.318951+0000","last_became_peered":"2023-10-06T02:45:23.318951+0000","last_unstale":"2023-10-06T02:45:27.334989+0000","last_undegraded":"2023-10-06T02:45:27.334989+0000","last_fullsized":"2023-10-06T02:45:27.334989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:34.946622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325697+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:45:27.325697+0000","last_peered":"2023-10-06T02:45:27.325697+0000","last_clean":"2023-10-06T02:45:27.325697+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:45:27.325697+0000","last_undegraded":"2023-10-06T02:45:27.325697+0000","last_fullsized":"2023-10-06T02:45:27.325697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:28.296097+0000","last_change":"2023-10-06T02:45:22.799764+0000","last_active":"2023-10-06T02:45:28.296097+0000","last_peered":"2023-10-06T02:45:28.296097+0000","last_clean":"2023-10-06T02:45:28.296097+0000","last_became_active":"2023-10-06T02:45:22.798917+0000","last_became_peered":"2023-10-06T02:45:22.798917+0000","last_unstale":"2023-10-06T02:45:28.296097+0000","last_undegraded":"2023-10-06T02:45:28.296097+0000","last_fullsized":"2023-10-06T02:45:28.296097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353993+0000","last_change":"2023-10-06T02:45:27.350164+0000","last_active":"2023-10-06T02:45:27.353993+0000","last_peered":"2023-10-06T02:45:27.353993+0000","last_clean":"2023-10-06T02:45:27.353993+0000","last_became_active":"2023-10-06T02:45:27.349731+0000","last_became_peered":"2023-10-06T02:45:27.349731+0000","last_unstale":"2023-10-06T02:45:27.353993+0000","last_undegraded":"2023-10-06T02:45:27.353993+0000","last_fullsized":"2023-10-06T02:45:27.353993+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:07.179714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332977+0000","last_change":"2023-10-06T02:45:23.324382+0000","last_active":"2023-10-06T02:45:27.332977+0000","last_peered":"2023-10-06T02:45:27.332977+0000","last_clean":"2023-10-06T02:45:27.332977+0000","last_became_active":"2023-10-06T02:45:23.324017+0000","last_became_peered":"2023-10-06T02:45:23.324017+0000","last_unstale":"2023-10-06T02:45:27.332977+0000","last_undegraded":"2023-10-06T02:45:27.332977+0000","last_fullsized":"2023-10-06T02:45:27.332977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:01.347856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340460+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:45:27.340460+0000","last_peered":"2023-10-06T02:45:27.340460+0000","last_clean":"2023-10-06T02:45:27.340460+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:45:27.340460+0000","last_undegraded":"2023-10-06T02:45:27.340460+0000","last_fullsized":"2023-10-06T02:45:27.340460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333588+0000","last_change":"2023-10-06T02:45:22.807333+0000","last_active":"2023-10-06T02:45:27.333588+0000","last_peered":"2023-10-06T02:45:27.333588+0000","last_clean":"2023-10-06T02:45:27.333588+0000","last_became_active":"2023-10-06T02:45:22.805988+0000","last_became_peered":"2023-10-06T02:45:22.805988+0000","last_unstale":"2023-10-06T02:45:27.333588+0000","last_undegraded":"2023-10-06T02:45:27.333588+0000","last_fullsized":"2023-10-06T02:45:27.333588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:41.870262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353464+0000","last_change":"2023-10-06T02:45:27.347859+0000","last_active":"2023-10-06T02:45:27.353464+0000","last_peered":"2023-10-06T02:45:27.353464+0000","last_clean":"2023-10-06T02:45:27.353464+0000","last_became_active":"2023-10-06T02:45:27.347418+0000","last_became_peered":"2023-10-06T02:45:27.347418+0000","last_unstale":"2023-10-06T02:45:27.353464+0000","last_undegraded":"2023-10-06T02:45:27.353464+0000","last_fullsized":"2023-10-06T02:45:27.353464+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:17.541971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334956+0000","last_change":"2023-10-06T02:45:23.320296+0000","last_active":"2023-10-06T02:45:27.334956+0000","last_peered":"2023-10-06T02:45:27.334956+0000","last_clean":"2023-10-06T02:45:27.334956+0000","last_became_active":"2023-10-06T02:45:23.319983+0000","last_became_peered":"2023-10-06T02:45:23.319983+0000","last_unstale":"2023-10-06T02:45:27.334956+0000","last_undegraded":"2023-10-06T02:45:27.334956+0000","last_fullsized":"2023-10-06T02:45:27.334956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333252+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:45:27.333252+0000","last_peered":"2023-10-06T02:45:27.333252+0000","last_clean":"2023-10-06T02:45:27.333252+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:45:27.333252+0000","last_undegraded":"2023-10-06T02:45:27.333252+0000","last_fullsized":"2023-10-06T02:45:27.333252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336685+0000","last_change":"2023-10-06T02:45:22.804019+0000","last_active":"2023-10-06T02:45:27.336685+0000","last_peered":"2023-10-06T02:45:27.336685+0000","last_clean":"2023-10-06T02:45:27.336685+0000","last_became_active":"2023-10-06T02:45:22.803524+0000","last_became_peered":"2023-10-06T02:45:22.803524+0000","last_unstale":"2023-10-06T02:45:27.336685+0000","last_undegraded":"2023-10-06T02:45:27.336685+0000","last_fullsized":"2023-10-06T02:45:27.336685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353592+0000","last_change":"2023-10-06T02:45:27.344027+0000","last_active":"2023-10-06T02:45:27.353592+0000","last_peered":"2023-10-06T02:45:27.353592+0000","last_clean":"2023-10-06T02:45:27.353592+0000","last_became_active":"2023-10-06T02:45:27.343450+0000","last_became_peered":"2023-10-06T02:45:27.343450+0000","last_unstale":"2023-10-06T02:45:27.353592+0000","last_undegraded":"2023-10-06T02:45:27.353592+0000","last_fullsized":"2023-10-06T02:45:27.353592+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:42.951337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336038+0000","last_change":"2023-10-06T02:45:23.319757+0000","last_active":"2023-10-06T02:45:27.336038+0000","last_peered":"2023-10-06T02:45:27.336038+0000","last_clean":"2023-10-06T02:45:27.336038+0000","last_became_active":"2023-10-06T02:45:23.319356+0000","last_became_peered":"2023-10-06T02:45:23.319356+0000","last_unstale":"2023-10-06T02:45:27.336038+0000","last_undegraded":"2023-10-06T02:45:27.336038+0000","last_fullsized":"2023-10-06T02:45:27.336038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337310+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:45:27.337310+0000","last_peered":"2023-10-06T02:45:27.337310+0000","last_clean":"2023-10-06T02:45:27.337310+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:45:27.337310+0000","last_undegraded":"2023-10-06T02:45:27.337310+0000","last_fullsized":"2023-10-06T02:45:27.337310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341535+0000","last_change":"2023-10-06T02:45:22.808675+0000","last_active":"2023-10-06T02:45:27.341535+0000","last_peered":"2023-10-06T02:45:27.341535+0000","last_clean":"2023-10-06T02:45:27.341535+0000","last_became_active":"2023-10-06T02:45:22.807693+0000","last_became_peered":"2023-10-06T02:45:22.807693+0000","last_unstale":"2023-10-06T02:45:27.341535+0000","last_undegraded":"2023-10-06T02:45:27.341535+0000","last_fullsized":"2023-10-06T02:45:27.341535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356445+0000","last_change":"2023-10-06T02:45:27.352852+0000","last_active":"2023-10-06T02:45:27.356445+0000","last_peered":"2023-10-06T02:45:27.356445+0000","last_clean":"2023-10-06T02:45:27.356445+0000","last_became_active":"2023-10-06T02:45:27.352303+0000","last_became_peered":"2023-10-06T02:45:27.352303+0000","last_unstale":"2023-10-06T02:45:27.356445+0000","last_undegraded":"2023-10-06T02:45:27.356445+0000","last_fullsized":"2023-10-06T02:45:27.356445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:21.854256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336597+0000","last_change":"2023-10-06T02:45:23.317570+0000","last_active":"2023-10-06T02:45:27.336597+0000","last_peered":"2023-10-06T02:45:27.336597+0000","last_clean":"2023-10-06T02:45:27.336597+0000","last_became_active":"2023-10-06T02:45:23.316419+0000","last_became_peered":"2023-10-06T02:45:23.316419+0000","last_unstale":"2023-10-06T02:45:27.336597+0000","last_undegraded":"2023-10-06T02:45:27.336597+0000","last_fullsized":"2023-10-06T02:45:27.336597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332685+0000","last_change":"2023-10-06T02:45:22.799727+0000","last_active":"2023-10-06T02:45:27.332685+0000","last_peered":"2023-10-06T02:45:27.332685+0000","last_clean":"2023-10-06T02:45:27.332685+0000","last_became_active":"2023-10-06T02:45:22.799276+0000","last_became_peered":"2023-10-06T02:45:22.799276+0000","last_unstale":"2023-10-06T02:45:27.332685+0000","last_undegraded":"2023-10-06T02:45:27.332685+0000","last_fullsized":"2023-10-06T02:45:27.332685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340308+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:45:27.340308+0000","last_peered":"2023-10-06T02:45:27.340308+0000","last_clean":"2023-10-06T02:45:27.340308+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:45:27.340308+0000","last_undegraded":"2023-10-06T02:45:27.340308+0000","last_fullsized":"2023-10-06T02:45:27.340308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353669+0000","last_change":"2023-10-06T02:45:27.350898+0000","last_active":"2023-10-06T02:45:27.353669+0000","last_peered":"2023-10-06T02:45:27.353669+0000","last_clean":"2023-10-06T02:45:27.353669+0000","last_became_active":"2023-10-06T02:45:27.350489+0000","last_became_peered":"2023-10-06T02:45:27.350489+0000","last_unstale":"2023-10-06T02:45:27.353669+0000","last_undegraded":"2023-10-06T02:45:27.353669+0000","last_fullsized":"2023-10-06T02:45:27.353669+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340516+0000","last_change":"2023-10-06T02:45:23.320581+0000","last_active":"2023-10-06T02:45:27.340516+0000","last_peered":"2023-10-06T02:45:27.340516+0000","last_clean":"2023-10-06T02:45:27.340516+0000","last_became_active":"2023-10-06T02:45:23.320253+0000","last_became_peered":"2023-10-06T02:45:23.320253+0000","last_unstale":"2023-10-06T02:45:27.340516+0000","last_undegraded":"2023-10-06T02:45:27.340516+0000","last_fullsized":"2023-10-06T02:45:27.340516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:15.466869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334594+0000","last_change":"2023-10-06T02:45:22.803922+0000","last_active":"2023-10-06T02:45:27.334594+0000","last_peered":"2023-10-06T02:45:27.334594+0000","last_clean":"2023-10-06T02:45:27.334594+0000","last_became_active":"2023-10-06T02:45:22.803487+0000","last_became_peered":"2023-10-06T02:45:22.803487+0000","last_unstale":"2023-10-06T02:45:27.334594+0000","last_undegraded":"2023-10-06T02:45:27.334594+0000","last_fullsized":"2023-10-06T02:45:27.334594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319371+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:45:27.319371+0000","last_peered":"2023-10-06T02:45:27.319371+0000","last_clean":"2023-10-06T02:45:27.319371+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:45:27.319371+0000","last_undegraded":"2023-10-06T02:45:27.319371+0000","last_fullsized":"2023-10-06T02:45:27.319371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350330+0000","last_change":"2023-10-06T02:45:27.341548+0000","last_active":"2023-10-06T02:45:27.350330+0000","last_peered":"2023-10-06T02:45:27.350330+0000","last_clean":"2023-10-06T02:45:27.350330+0000","last_became_active":"2023-10-06T02:45:27.341116+0000","last_became_peered":"2023-10-06T02:45:27.341116+0000","last_unstale":"2023-10-06T02:45:27.350330+0000","last_undegraded":"2023-10-06T02:45:27.350330+0000","last_fullsized":"2023-10-06T02:45:27.350330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:12.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334501+0000","last_change":"2023-10-06T02:45:23.313236+0000","last_active":"2023-10-06T02:45:27.334501+0000","last_peered":"2023-10-06T02:45:27.334501+0000","last_clean":"2023-10-06T02:45:27.334501+0000","last_became_active":"2023-10-06T02:45:23.312943+0000","last_became_peered":"2023-10-06T02:45:23.312943+0000","last_unstale":"2023-10-06T02:45:27.334501+0000","last_undegraded":"2023-10-06T02:45:27.334501+0000","last_fullsized":"2023-10-06T02:45:27.334501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333980+0000","last_change":"2023-10-06T02:45:22.801183+0000","last_active":"2023-10-06T02:45:27.333980+0000","last_peered":"2023-10-06T02:45:27.333980+0000","last_clean":"2023-10-06T02:45:27.333980+0000","last_became_active":"2023-10-06T02:45:22.800872+0000","last_became_peered":"2023-10-06T02:45:22.800872+0000","last_unstale":"2023-10-06T02:45:27.333980+0000","last_undegraded":"2023-10-06T02:45:27.333980+0000","last_fullsized":"2023-10-06T02:45:27.333980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331616+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:45:27.331616+0000","last_peered":"2023-10-06T02:45:27.331616+0000","last_clean":"2023-10-06T02:45:27.331616+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:45:27.331616+0000","last_undegraded":"2023-10-06T02:45:27.331616+0000","last_fullsized":"2023-10-06T02:45:27.331616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357435+0000","last_change":"2023-10-06T02:45:27.354883+0000","last_active":"2023-10-06T02:45:27.357435+0000","last_peered":"2023-10-06T02:45:27.357435+0000","last_clean":"2023-10-06T02:45:27.357435+0000","last_became_active":"2023-10-06T02:45:27.353904+0000","last_became_peered":"2023-10-06T02:45:27.353904+0000","last_unstale":"2023-10-06T02:45:27.357435+0000","last_undegraded":"2023-10-06T02:45:27.357435+0000","last_fullsized":"2023-10-06T02:45:27.357435+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:54.845793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325620+0000","last_change":"2023-10-06T02:45:23.311475+0000","last_active":"2023-10-06T02:45:27.325620+0000","last_peered":"2023-10-06T02:45:27.325620+0000","last_clean":"2023-10-06T02:45:27.325620+0000","last_became_active":"2023-10-06T02:45:23.311183+0000","last_became_peered":"2023-10-06T02:45:23.311183+0000","last_unstale":"2023-10-06T02:45:27.325620+0000","last_undegraded":"2023-10-06T02:45:27.325620+0000","last_fullsized":"2023-10-06T02:45:27.325620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:38.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331867+0000","last_change":"2023-10-06T02:45:22.800538+0000","last_active":"2023-10-06T02:45:27.331867+0000","last_peered":"2023-10-06T02:45:27.331867+0000","last_clean":"2023-10-06T02:45:27.331867+0000","last_became_active":"2023-10-06T02:45:22.800251+0000","last_became_peered":"2023-10-06T02:45:22.800251+0000","last_unstale":"2023-10-06T02:45:27.331867+0000","last_undegraded":"2023-10-06T02:45:27.331867+0000","last_fullsized":"2023-10-06T02:45:27.331867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:35.553098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340818+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:45:27.340818+0000","last_peered":"2023-10-06T02:45:27.340818+0000","last_clean":"2023-10-06T02:45:27.340818+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:45:27.340818+0000","last_undegraded":"2023-10-06T02:45:27.340818+0000","last_fullsized":"2023-10-06T02:45:27.340818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331708+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:45:27.331708+0000","last_peered":"2023-10-06T02:45:27.331708+0000","last_clean":"2023-10-06T02:45:27.331708+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:45:27.331708+0000","last_undegraded":"2023-10-06T02:45:27.331708+0000","last_fullsized":"2023-10-06T02:45:27.331708+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351388+0000","last_change":"2023-10-06T02:45:27.349035+0000","last_active":"2023-10-06T02:45:27.351388+0000","last_peered":"2023-10-06T02:45:27.351388+0000","last_clean":"2023-10-06T02:45:27.351388+0000","last_became_active":"2023-10-06T02:45:27.348608+0000","last_became_peered":"2023-10-06T02:45:27.348608+0000","last_unstale":"2023-10-06T02:45:27.351388+0000","last_undegraded":"2023-10-06T02:45:27.351388+0000","last_fullsized":"2023-10-06T02:45:27.351388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:18.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340259+0000","last_change":"2023-10-06T02:45:23.310743+0000","last_active":"2023-10-06T02:45:27.340259+0000","last_peered":"2023-10-06T02:45:27.340259+0000","last_clean":"2023-10-06T02:45:27.340259+0000","last_became_active":"2023-10-06T02:45:23.310401+0000","last_became_peered":"2023-10-06T02:45:23.310401+0000","last_unstale":"2023-10-06T02:45:27.340259+0000","last_undegraded":"2023-10-06T02:45:27.340259+0000","last_fullsized":"2023-10-06T02:45:27.340259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333057+0000","last_change":"2023-10-06T02:45:22.807407+0000","last_active":"2023-10-06T02:45:27.333057+0000","last_peered":"2023-10-06T02:45:27.333057+0000","last_clean":"2023-10-06T02:45:27.333057+0000","last_became_active":"2023-10-06T02:45:22.804974+0000","last_became_peered":"2023-10-06T02:45:22.804974+0000","last_unstale":"2023-10-06T02:45:27.333057+0000","last_undegraded":"2023-10-06T02:45:27.333057+0000","last_fullsized":"2023-10-06T02:45:27.333057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:15.722413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351625+0000","last_change":"2023-10-06T02:45:27.346225+0000","last_active":"2023-10-06T02:45:27.351625+0000","last_peered":"2023-10-06T02:45:27.351625+0000","last_clean":"2023-10-06T02:45:27.351625+0000","last_became_active":"2023-10-06T02:45:27.345701+0000","last_became_peered":"2023-10-06T02:45:27.345701+0000","last_unstale":"2023-10-06T02:45:27.351625+0000","last_undegraded":"2023-10-06T02:45:27.351625+0000","last_fullsized":"2023-10-06T02:45:27.351625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:56.453178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334551+0000","last_change":"2023-10-06T02:45:23.310126+0000","last_active":"2023-10-06T02:45:27.334551+0000","last_peered":"2023-10-06T02:45:27.334551+0000","last_clean":"2023-10-06T02:45:27.334551+0000","last_became_active":"2023-10-06T02:45:23.309805+0000","last_became_peered":"2023-10-06T02:45:23.309805+0000","last_unstale":"2023-10-06T02:45:27.334551+0000","last_undegraded":"2023-10-06T02:45:27.334551+0000","last_fullsized":"2023-10-06T02:45:27.334551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336853+0000","last_change":"2023-10-06T02:45:22.804564+0000","last_active":"2023-10-06T02:45:27.336853+0000","last_peered":"2023-10-06T02:45:27.336853+0000","last_clean":"2023-10-06T02:45:27.336853+0000","last_became_active":"2023-10-06T02:45:22.804286+0000","last_became_peered":"2023-10-06T02:45:22.804286+0000","last_unstale":"2023-10-06T02:45:27.336853+0000","last_undegraded":"2023-10-06T02:45:27.336853+0000","last_fullsized":"2023-10-06T02:45:27.336853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357099+0000","last_change":"2023-10-06T02:45:27.354769+0000","last_active":"2023-10-06T02:45:27.357099+0000","last_peered":"2023-10-06T02:45:27.357099+0000","last_clean":"2023-10-06T02:45:27.357099+0000","last_became_active":"2023-10-06T02:45:27.352768+0000","last_became_peered":"2023-10-06T02:45:27.352768+0000","last_unstale":"2023-10-06T02:45:27.357099+0000","last_undegraded":"2023-10-06T02:45:27.357099+0000","last_fullsized":"2023-10-06T02:45:27.357099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336239+0000","last_change":"2023-10-06T02:45:23.310059+0000","last_active":"2023-10-06T02:45:27.336239+0000","last_peered":"2023-10-06T02:45:27.336239+0000","last_clean":"2023-10-06T02:45:27.336239+0000","last_became_active":"2023-10-06T02:45:23.309667+0000","last_became_peered":"2023-10-06T02:45:23.309667+0000","last_unstale":"2023-10-06T02:45:27.336239+0000","last_undegraded":"2023-10-06T02:45:27.336239+0000","last_fullsized":"2023-10-06T02:45:27.336239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:28.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.326136+0000","last_change":"2023-10-06T02:45:22.795986+0000","last_active":"2023-10-06T02:45:27.326136+0000","last_peered":"2023-10-06T02:45:27.326136+0000","last_clean":"2023-10-06T02:45:27.326136+0000","last_became_active":"2023-10-06T02:45:22.795649+0000","last_became_peered":"2023-10-06T02:45:22.795649+0000","last_unstale":"2023-10-06T02:45:27.326136+0000","last_undegraded":"2023-10-06T02:45:27.326136+0000","last_fullsized":"2023-10-06T02:45:27.326136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:52.473292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360933+0000","last_change":"2023-10-06T02:45:27.347788+0000","last_active":"2023-10-06T02:45:27.360933+0000","last_peered":"2023-10-06T02:45:27.360933+0000","last_clean":"2023-10-06T02:45:27.360933+0000","last_became_active":"2023-10-06T02:45:27.347192+0000","last_became_peered":"2023-10-06T02:45:27.347192+0000","last_unstale":"2023-10-06T02:45:27.360933+0000","last_undegraded":"2023-10-06T02:45:27.360933+0000","last_fullsized":"2023-10-06T02:45:27.360933+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336639+0000","last_change":"2023-10-06T02:45:23.307192+0000","last_active":"2023-10-06T02:45:27.336639+0000","last_peered":"2023-10-06T02:45:27.336639+0000","last_clean":"2023-10-06T02:45:27.336639+0000","last_became_active":"2023-10-06T02:45:23.306642+0000","last_became_peered":"2023-10-06T02:45:23.306642+0000","last_unstale":"2023-10-06T02:45:27.336639+0000","last_undegraded":"2023-10-06T02:45:27.336639+0000","last_fullsized":"2023-10-06T02:45:27.336639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:14.468949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333926+0000","last_change":"2023-10-06T02:45:22.811906+0000","last_active":"2023-10-06T02:45:27.333926+0000","last_peered":"2023-10-06T02:45:27.333926+0000","last_clean":"2023-10-06T02:45:27.333926+0000","last_became_active":"2023-10-06T02:45:22.811602+0000","last_became_peered":"2023-10-06T02:45:22.811602+0000","last_unstale":"2023-10-06T02:45:27.333926+0000","last_undegraded":"2023-10-06T02:45:27.333926+0000","last_fullsized":"2023-10-06T02:45:27.333926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.090764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.349516+0000","last_change":"2023-10-06T02:45:27.344016+0000","last_active":"2023-10-06T02:45:27.349516+0000","last_peered":"2023-10-06T02:45:27.349516+0000","last_clean":"2023-10-06T02:45:27.349516+0000","last_became_active":"2023-10-06T02:45:27.342950+0000","last_became_peered":"2023-10-06T02:45:27.342950+0000","last_unstale":"2023-10-06T02:45:27.349516+0000","last_undegraded":"2023-10-06T02:45:27.349516+0000","last_fullsized":"2023-10-06T02:45:27.349516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:25.726296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334869+0000","last_change":"2023-10-06T02:45:23.321013+0000","last_active":"2023-10-06T02:45:27.334869+0000","last_peered":"2023-10-06T02:45:27.334869+0000","last_clean":"2023-10-06T02:45:27.334869+0000","last_became_active":"2023-10-06T02:45:23.320682+0000","last_became_peered":"2023-10-06T02:45:23.320682+0000","last_unstale":"2023-10-06T02:45:27.334869+0000","last_undegraded":"2023-10-06T02:45:27.334869+0000","last_fullsized":"2023-10-06T02:45:27.334869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341800+0000","last_change":"2023-10-06T02:45:22.806416+0000","last_active":"2023-10-06T02:45:27.341800+0000","last_peered":"2023-10-06T02:45:27.341800+0000","last_clean":"2023-10-06T02:45:27.341800+0000","last_became_active":"2023-10-06T02:45:22.806087+0000","last_became_peered":"2023-10-06T02:45:22.806087+0000","last_unstale":"2023-10-06T02:45:27.341800+0000","last_undegraded":"2023-10-06T02:45:27.341800+0000","last_fullsized":"2023-10-06T02:45:27.341800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357937+0000","last_change":"2023-10-06T02:45:27.350897+0000","last_active":"2023-10-06T02:45:27.357937+0000","last_peered":"2023-10-06T02:45:27.357937+0000","last_clean":"2023-10-06T02:45:27.357937+0000","last_became_active":"2023-10-06T02:45:27.350497+0000","last_became_peered":"2023-10-06T02:45:27.350497+0000","last_unstale":"2023-10-06T02:45:27.357937+0000","last_undegraded":"2023-10-06T02:45:27.357937+0000","last_fullsized":"2023-10-06T02:45:27.357937+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:38.481448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325860+0000","last_change":"2023-10-06T02:45:23.313267+0000","last_active":"2023-10-06T02:45:27.325860+0000","last_peered":"2023-10-06T02:45:27.325860+0000","last_clean":"2023-10-06T02:45:27.325860+0000","last_became_active":"2023-10-06T02:45:23.312886+0000","last_became_peered":"2023-10-06T02:45:23.312886+0000","last_unstale":"2023-10-06T02:45:27.325860+0000","last_undegraded":"2023-10-06T02:45:27.325860+0000","last_fullsized":"2023-10-06T02:45:27.325860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:30.923831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333106+0000","last_change":"2023-10-06T02:45:22.809724+0000","last_active":"2023-10-06T02:45:27.333106+0000","last_peered":"2023-10-06T02:45:27.333106+0000","last_clean":"2023-10-06T02:45:27.333106+0000","last_became_active":"2023-10-06T02:45:22.809451+0000","last_became_peered":"2023-10-06T02:45:22.809451+0000","last_unstale":"2023-10-06T02:45:27.333106+0000","last_undegraded":"2023-10-06T02:45:27.333106+0000","last_fullsized":"2023-10-06T02:45:27.333106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:10.054721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.348579+0000","last_change":"2023-10-06T02:45:27.343402+0000","last_active":"2023-10-06T02:45:27.348579+0000","last_peered":"2023-10-06T02:45:27.348579+0000","last_clean":"2023-10-06T02:45:27.348579+0000","last_became_active":"2023-10-06T02:45:27.342892+0000","last_became_peered":"2023-10-06T02:45:27.342892+0000","last_unstale":"2023-10-06T02:45:27.348579+0000","last_undegraded":"2023-10-06T02:45:27.348579+0000","last_fullsized":"2023-10-06T02:45:27.348579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:05.116878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334800+0000","last_change":"2023-10-06T02:45:23.313977+0000","last_active":"2023-10-06T02:45:27.334800+0000","last_peered":"2023-10-06T02:45:27.334800+0000","last_clean":"2023-10-06T02:45:27.334800+0000","last_became_active":"2023-10-06T02:45:23.313656+0000","last_became_peered":"2023-10-06T02:45:23.313656+0000","last_unstale":"2023-10-06T02:45:27.334800+0000","last_undegraded":"2023-10-06T02:45:27.334800+0000","last_fullsized":"2023-10-06T02:45:27.334800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:52.328524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334307+0000","last_change":"2023-10-06T02:45:22.809096+0000","last_active":"2023-10-06T02:45:27.334307+0000","last_peered":"2023-10-06T02:45:27.334307+0000","last_clean":"2023-10-06T02:45:27.334307+0000","last_became_active":"2023-10-06T02:45:22.807367+0000","last_became_peered":"2023-10-06T02:45:22.807367+0000","last_unstale":"2023-10-06T02:45:27.334307+0000","last_undegraded":"2023-10-06T02:45:27.334307+0000","last_fullsized":"2023-10-06T02:45:27.334307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357950+0000","last_change":"2023-10-06T02:45:27.349954+0000","last_active":"2023-10-06T02:45:27.357950+0000","last_peered":"2023-10-06T02:45:27.357950+0000","last_clean":"2023-10-06T02:45:27.357950+0000","last_became_active":"2023-10-06T02:45:27.344601+0000","last_became_peered":"2023-10-06T02:45:27.344601+0000","last_unstale":"2023-10-06T02:45:27.357950+0000","last_undegraded":"2023-10-06T02:45:27.357950+0000","last_fullsized":"2023-10-06T02:45:27.357950+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:55.863350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340458+0000","last_change":"2023-10-06T02:45:23.325407+0000","last_active":"2023-10-06T02:45:27.340458+0000","last_peered":"2023-10-06T02:45:27.340458+0000","last_clean":"2023-10-06T02:45:27.340458+0000","last_became_active":"2023-10-06T02:45:23.325027+0000","last_became_peered":"2023-10-06T02:45:23.325027+0000","last_unstale":"2023-10-06T02:45:27.340458+0000","last_undegraded":"2023-10-06T02:45:27.340458+0000","last_fullsized":"2023-10-06T02:45:27.340458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334293+0000","last_change":"2023-10-06T02:45:22.807221+0000","last_active":"2023-10-06T02:45:27.334293+0000","last_peered":"2023-10-06T02:45:27.334293+0000","last_clean":"2023-10-06T02:45:27.334293+0000","last_became_active":"2023-10-06T02:45:22.806899+0000","last_became_peered":"2023-10-06T02:45:22.806899+0000","last_unstale":"2023-10-06T02:45:27.334293+0000","last_undegraded":"2023-10-06T02:45:27.334293+0000","last_fullsized":"2023-10-06T02:45:27.334293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356009+0000","last_change":"2023-10-06T02:45:27.352601+0000","last_active":"2023-10-06T02:45:27.356009+0000","last_peered":"2023-10-06T02:45:27.356009+0000","last_clean":"2023-10-06T02:45:27.356009+0000","last_became_active":"2023-10-06T02:45:27.352177+0000","last_became_peered":"2023-10-06T02:45:27.352177+0000","last_unstale":"2023-10-06T02:45:27.356009+0000","last_undegraded":"2023-10-06T02:45:27.356009+0000","last_fullsized":"2023-10-06T02:45:27.356009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:57.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357070+0000","last_change":"2023-10-06T02:45:27.354742+0000","last_active":"2023-10-06T02:45:27.357070+0000","last_peered":"2023-10-06T02:45:27.357070+0000","last_clean":"2023-10-06T02:45:27.357070+0000","last_became_active":"2023-10-06T02:45:27.354154+0000","last_became_peered":"2023-10-06T02:45:27.354154+0000","last_unstale":"2023-10-06T02:45:27.357070+0000","last_undegraded":"2023-10-06T02:45:27.357070+0000","last_fullsized":"2023-10-06T02:45:27.357070+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:03.091381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.327140+0000","last_change":"2023-10-06T02:45:22.802084+0000","last_active":"2023-10-06T02:45:27.327140+0000","last_peered":"2023-10-06T02:45:27.327140+0000","last_clean":"2023-10-06T02:45:27.327140+0000","last_became_active":"2023-10-06T02:45:22.801685+0000","last_became_peered":"2023-10-06T02:45:22.801685+0000","last_unstale":"2023-10-06T02:45:27.327140+0000","last_undegraded":"2023-10-06T02:45:27.327140+0000","last_fullsized":"2023-10-06T02:45:27.327140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:31.821785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333425+0000","last_change":"2023-10-06T02:45:23.323734+0000","last_active":"2023-10-06T02:45:27.333425+0000","last_peered":"2023-10-06T02:45:27.333425+0000","last_clean":"2023-10-06T02:45:27.333425+0000","last_became_active":"2023-10-06T02:45:23.323390+0000","last_became_peered":"2023-10-06T02:45:23.323390+0000","last_unstale":"2023-10-06T02:45:27.333425+0000","last_undegraded":"2023-10-06T02:45:27.333425+0000","last_fullsized":"2023-10-06T02:45:27.333425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:09:04.032805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350027+0000","last_change":"2023-10-06T02:45:27.346307+0000","last_active":"2023-10-06T02:45:27.350027+0000","last_peered":"2023-10-06T02:45:27.350027+0000","last_clean":"2023-10-06T02:45:27.350027+0000","last_became_active":"2023-10-06T02:45:27.345807+0000","last_became_peered":"2023-10-06T02:45:27.345807+0000","last_unstale":"2023-10-06T02:45:27.350027+0000","last_undegraded":"2023-10-06T02:45:27.350027+0000","last_fullsized":"2023-10-06T02:45:27.350027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.633950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332194+0000","last_change":"2023-10-06T02:45:22.809079+0000","last_active":"2023-10-06T02:45:27.332194+0000","last_peered":"2023-10-06T02:45:27.332194+0000","last_clean":"2023-10-06T02:45:27.332194+0000","last_became_active":"2023-10-06T02:45:22.807326+0000","last_became_peered":"2023-10-06T02:45:22.807326+0000","last_unstale":"2023-10-06T02:45:27.332194+0000","last_undegraded":"2023-10-06T02:45:27.332194+0000","last_fullsized":"2023-10-06T02:45:27.332194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:32.124040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334582+0000","last_change":"2023-10-06T02:45:23.313113+0000","last_active":"2023-10-06T02:45:27.334582+0000","last_peered":"2023-10-06T02:45:27.334582+0000","last_clean":"2023-10-06T02:45:27.334582+0000","last_became_active":"2023-10-06T02:45:23.312797+0000","last_became_peered":"2023-10-06T02:45:23.312797+0000","last_unstale":"2023-10-06T02:45:27.334582+0000","last_undegraded":"2023-10-06T02:45:27.334582+0000","last_fullsized":"2023-10-06T02:45:27.334582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334672+0000","last_change":"2023-10-06T02:45:22.804236+0000","last_active":"2023-10-06T02:45:27.334672+0000","last_peered":"2023-10-06T02:45:27.334672+0000","last_clean":"2023-10-06T02:45:27.334672+0000","last_became_active":"2023-10-06T02:45:22.802766+0000","last_became_peered":"2023-10-06T02:45:22.802766+0000","last_unstale":"2023-10-06T02:45:27.334672+0000","last_undegraded":"2023-10-06T02:45:27.334672+0000","last_fullsized":"2023-10-06T02:45:27.334672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341457+0000","last_change":"2023-10-06T02:45:23.324554+0000","last_active":"2023-10-06T02:45:27.341457+0000","last_peered":"2023-10-06T02:45:27.341457+0000","last_clean":"2023-10-06T02:45:27.341457+0000","last_became_active":"2023-10-06T02:45:23.324267+0000","last_became_peered":"2023-10-06T02:45:23.324267+0000","last_unstale":"2023-10-06T02:45:27.341457+0000","last_undegraded":"2023-10-06T02:45:27.341457+0000","last_fullsized":"2023-10-06T02:45:27.341457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354813+0000","last_change":"2023-10-06T02:45:27.346623+0000","last_active":"2023-10-06T02:45:27.354813+0000","last_peered":"2023-10-06T02:45:27.354813+0000","last_clean":"2023-10-06T02:45:27.354813+0000","last_became_active":"2023-10-06T02:45:27.346022+0000","last_became_peered":"2023-10-06T02:45:27.346022+0000","last_unstale":"2023-10-06T02:45:27.354813+0000","last_undegraded":"2023-10-06T02:45:27.354813+0000","last_fullsized":"2023-10-06T02:45:27.354813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:40.724648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332912+0000","last_change":"2023-10-06T02:45:22.805769+0000","last_active":"2023-10-06T02:45:27.332912+0000","last_peered":"2023-10-06T02:45:27.332912+0000","last_clean":"2023-10-06T02:45:27.332912+0000","last_became_active":"2023-10-06T02:45:22.805441+0000","last_became_peered":"2023-10-06T02:45:22.805441+0000","last_unstale":"2023-10-06T02:45:27.332912+0000","last_undegraded":"2023-10-06T02:45:27.332912+0000","last_fullsized":"2023-10-06T02:45:27.332912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:41.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.338579+0000","last_change":"2023-10-06T02:45:23.312634+0000","last_active":"2023-10-06T02:45:27.338579+0000","last_peered":"2023-10-06T02:45:27.338579+0000","last_clean":"2023-10-06T02:45:27.338579+0000","last_became_active":"2023-10-06T02:45:23.312292+0000","last_became_peered":"2023-10-06T02:45:23.312292+0000","last_unstale":"2023-10-06T02:45:27.338579+0000","last_undegraded":"2023-10-06T02:45:27.338579+0000","last_fullsized":"2023-10-06T02:45:27.338579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:05.795770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357104+0000","last_change":"2023-10-06T02:45:27.354353+0000","last_active":"2023-10-06T02:45:27.357104+0000","last_peered":"2023-10-06T02:45:27.357104+0000","last_clean":"2023-10-06T02:45:27.357104+0000","last_became_active":"2023-10-06T02:45:27.353428+0000","last_became_peered":"2023-10-06T02:45:27.353428+0000","last_unstale":"2023-10-06T02:45:27.357104+0000","last_undegraded":"2023-10-06T02:45:27.357104+0000","last_fullsized":"2023-10-06T02:45:27.357104+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.149006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325192+0000","last_change":"2023-10-06T02:45:23.309222+0000","last_active":"2023-10-06T02:45:27.325192+0000","last_peered":"2023-10-06T02:45:27.325192+0000","last_clean":"2023-10-06T02:45:27.325192+0000","last_became_active":"2023-10-06T02:45:23.308920+0000","last_became_peered":"2023-10-06T02:45:23.308920+0000","last_unstale":"2023-10-06T02:45:27.325192+0000","last_undegraded":"2023-10-06T02:45:27.325192+0000","last_fullsized":"2023-10-06T02:45:27.325192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:41.399143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329866+0000","last_change":"2023-10-06T02:45:23.312534+0000","last_active":"2023-10-06T02:45:27.329866+0000","last_peered":"2023-10-06T02:45:27.329866+0000","last_clean":"2023-10-06T02:45:27.329866+0000","last_became_active":"2023-10-06T02:45:23.312215+0000","last_became_peered":"2023-10-06T02:45:23.312215+0000","last_unstale":"2023-10-06T02:45:27.329866+0000","last_undegraded":"2023-10-06T02:45:27.329866+0000","last_fullsized":"2023-10-06T02:45:27.329866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:39.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336549+0000","last_change":"2023-10-06T02:45:23.305501+0000","last_active":"2023-10-06T02:45:27.336549+0000","last_peered":"2023-10-06T02:45:27.336549+0000","last_clean":"2023-10-06T02:45:27.336549+0000","last_became_active":"2023-10-06T02:45:23.305062+0000","last_became_peered":"2023-10-06T02:45:23.305062+0000","last_unstale":"2023-10-06T02:45:27.336549+0000","last_undegraded":"2023-10-06T02:45:27.336549+0000","last_fullsized":"2023-10-06T02:45:27.336549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333770+0000","last_change":"2023-10-06T02:45:23.324370+0000","last_active":"2023-10-06T02:45:27.333770+0000","last_peered":"2023-10-06T02:45:27.333770+0000","last_clean":"2023-10-06T02:45:27.333770+0000","last_became_active":"2023-10-06T02:45:23.324082+0000","last_became_peered":"2023-10-06T02:45:23.324082+0000","last_unstale":"2023-10-06T02:45:27.333770+0000","last_undegraded":"2023-10-06T02:45:27.333770+0000","last_fullsized":"2023-10-06T02:45:27.333770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:35.146269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337261+0000","last_change":"2023-10-06T02:45:23.322276+0000","last_active":"2023-10-06T02:45:27.337261+0000","last_peered":"2023-10-06T02:45:27.337261+0000","last_clean":"2023-10-06T02:45:27.337261+0000","last_became_active":"2023-10-06T02:45:23.321381+0000","last_became_peered":"2023-10-06T02:45:23.321381+0000","last_unstale":"2023-10-06T02:45:27.337261+0000","last_undegraded":"2023-10-06T02:45:27.337261+0000","last_fullsized":"2023-10-06T02:45:27.337261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341408+0000","last_change":"2023-10-06T02:45:23.324552+0000","last_active":"2023-10-06T02:45:27.341408+0000","last_peered":"2023-10-06T02:45:27.341408+0000","last_clean":"2023-10-06T02:45:27.341408+0000","last_became_active":"2023-10-06T02:45:23.324145+0000","last_became_peered":"2023-10-06T02:45:23.324145+0000","last_unstale":"2023-10-06T02:45:27.341408+0000","last_undegraded":"2023-10-06T02:45:27.341408+0000","last_fullsized":"2023-10-06T02:45:27.341408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340251+0000","last_change":"2023-10-06T02:45:23.316105+0000","last_active":"2023-10-06T02:45:27.340251+0000","last_peered":"2023-10-06T02:45:27.340251+0000","last_clean":"2023-10-06T02:45:27.340251+0000","last_became_active":"2023-10-06T02:45:23.315133+0000","last_became_peered":"2023-10-06T02:45:23.315133+0000","last_unstale":"2023-10-06T02:45:27.340251+0000","last_undegraded":"2023-10-06T02:45:27.340251+0000","last_fullsized":"2023-10-06T02:45:27.340251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.339997+0000","last_change":"2023-10-06T02:45:23.315937+0000","last_active":"2023-10-06T02:45:27.339997+0000","last_peered":"2023-10-06T02:45:27.339997+0000","last_clean":"2023-10-06T02:45:27.339997+0000","last_became_active":"2023-10-06T02:45:23.315541+0000","last_became_peered":"2023-10-06T02:45:23.315541+0000","last_unstale":"2023-10-06T02:45:27.339997+0000","last_undegraded":"2023-10-06T02:45:27.339997+0000","last_fullsized":"2023-10-06T02:45:27.339997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331530+0000","last_change":"2023-10-06T02:45:23.326576+0000","last_active":"2023-10-06T02:45:27.331530+0000","last_peered":"2023-10-06T02:45:27.331530+0000","last_clean":"2023-10-06T02:45:27.331530+0000","last_became_active":"2023-10-06T02:45:23.326298+0000","last_became_peered":"2023-10-06T02:45:23.326298+0000","last_unstale":"2023-10-06T02:45:27.331530+0000","last_undegraded":"2023-10-06T02:45:27.331530+0000","last_fullsized":"2023-10-06T02:45:27.331530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329766+0000","last_change":"2023-10-06T02:45:23.315079+0000","last_active":"2023-10-06T02:45:27.329766+0000","last_peered":"2023-10-06T02:45:27.329766+0000","last_clean":"2023-10-06T02:45:27.329766+0000","last_became_active":"2023-10-06T02:45:23.314780+0000","last_became_peered":"2023-10-06T02:45:23.314780+0000","last_unstale":"2023-10-06T02:45:27.329766+0000","last_undegraded":"2023-10-06T02:45:27.329766+0000","last_fullsized":"2023-10-06T02:45:27.329766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:22.869594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:23.314883+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:23.314594+0000","last_became_peered":"2023-10-06T02:45:23.314594+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332695+0000","last_change":"2023-10-06T02:45:23.324062+0000","last_active":"2023-10-06T02:45:27.332695+0000","last_peered":"2023-10-06T02:45:27.332695+0000","last_clean":"2023-10-06T02:45:27.332695+0000","last_became_active":"2023-10-06T02:45:23.323735+0000","last_became_peered":"2023-10-06T02:45:23.323735+0000","last_unstale":"2023-10-06T02:45:27.332695+0000","last_undegraded":"2023-10-06T02:45:27.332695+0000","last_fullsized":"2023-10-06T02:45:27.332695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336547+0000","last_change":"2023-10-06T02:45:23.306965+0000","last_active":"2023-10-06T02:45:27.336547+0000","last_peered":"2023-10-06T02:45:27.336547+0000","last_clean":"2023-10-06T02:45:27.336547+0000","last_became_active":"2023-10-06T02:45:23.306269+0000","last_became_peered":"2023-10-06T02:45:23.306269+0000","last_unstale":"2023-10-06T02:45:27.336547+0000","last_undegraded":"2023-10-06T02:45:27.336547+0000","last_fullsized":"2023-10-06T02:45:27.336547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320250+0000","last_change":"2023-10-06T02:45:23.305622+0000","last_active":"2023-10-06T02:45:27.320250+0000","last_peered":"2023-10-06T02:45:27.320250+0000","last_clean":"2023-10-06T02:45:27.320250+0000","last_became_active":"2023-10-06T02:45:23.305323+0000","last_became_peered":"2023-10-06T02:45:23.305323+0000","last_unstale":"2023-10-06T02:45:27.320250+0000","last_undegraded":"2023-10-06T02:45:27.320250+0000","last_fullsized":"2023-10-06T02:45:27.320250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:24.265950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334435+0000","last_change":"2023-10-06T02:45:23.311009+0000","last_active":"2023-10-06T02:45:27.334435+0000","last_peered":"2023-10-06T02:45:27.334435+0000","last_clean":"2023-10-06T02:45:27.334435+0000","last_became_active":"2023-10-06T02:45:23.310671+0000","last_became_peered":"2023-10-06T02:45:23.310671+0000","last_unstale":"2023-10-06T02:45:27.334435+0000","last_undegraded":"2023-10-06T02:45:27.334435+0000","last_fullsized":"2023-10-06T02:45:27.334435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:06.358484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334894+0000","last_change":"2023-10-06T02:45:23.312474+0000","last_active":"2023-10-06T02:45:27.334894+0000","last_peered":"2023-10-06T02:45:27.334894+0000","last_clean":"2023-10-06T02:45:27.334894+0000","last_became_active":"2023-10-06T02:45:23.312147+0000","last_became_peered":"2023-10-06T02:45:23.312147+0000","last_unstale":"2023-10-06T02:45:27.334894+0000","last_undegraded":"2023-10-06T02:45:27.334894+0000","last_fullsized":"2023-10-06T02:45:27.334894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330934+0000","last_change":"2023-10-06T02:45:23.314526+0000","last_active":"2023-10-06T02:45:27.330934+0000","last_peered":"2023-10-06T02:45:27.330934+0000","last_clean":"2023-10-06T02:45:27.330934+0000","last_became_active":"2023-10-06T02:45:23.314229+0000","last_became_peered":"2023-10-06T02:45:23.314229+0000","last_unstale":"2023-10-06T02:45:27.330934+0000","last_undegraded":"2023-10-06T02:45:27.330934+0000","last_fullsized":"2023-10-06T02:45:27.330934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:08.831291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332089+0000","last_change":"2023-10-06T02:45:23.326220+0000","last_active":"2023-10-06T02:45:27.332089+0000","last_peered":"2023-10-06T02:45:27.332089+0000","last_clean":"2023-10-06T02:45:27.332089+0000","last_became_active":"2023-10-06T02:45:23.325853+0000","last_became_peered":"2023-10-06T02:45:23.325853+0000","last_unstale":"2023-10-06T02:45:27.332089+0000","last_undegraded":"2023-10-06T02:45:27.332089+0000","last_fullsized":"2023-10-06T02:45:27.332089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340313+0000","last_change":"2023-10-06T02:45:23.319007+0000","last_active":"2023-10-06T02:45:27.340313+0000","last_peered":"2023-10-06T02:45:27.340313+0000","last_clean":"2023-10-06T02:45:27.340313+0000","last_became_active":"2023-10-06T02:45:23.318521+0000","last_became_peered":"2023-10-06T02:45:23.318521+0000","last_unstale":"2023-10-06T02:45:27.340313+0000","last_undegraded":"2023-10-06T02:45:27.340313+0000","last_fullsized":"2023-10-06T02:45:27.340313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319562+0000","last_change":"2023-10-06T02:45:23.308224+0000","last_active":"2023-10-06T02:45:27.319562+0000","last_peered":"2023-10-06T02:45:27.319562+0000","last_clean":"2023-10-06T02:45:27.319562+0000","last_became_active":"2023-10-06T02:45:23.307908+0000","last_became_peered":"2023-10-06T02:45:23.307908+0000","last_unstale":"2023-10-06T02:45:27.319562+0000","last_undegraded":"2023-10-06T02:45:27.319562+0000","last_fullsized":"2023-10-06T02:45:27.319562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:26.518844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319631+0000","last_change":"2023-10-06T02:45:23.312138+0000","last_active":"2023-10-06T02:45:27.319631+0000","last_peered":"2023-10-06T02:45:27.319631+0000","last_clean":"2023-10-06T02:45:27.319631+0000","last_became_active":"2023-10-06T02:45:23.311612+0000","last_became_peered":"2023-10-06T02:45:23.311612+0000","last_unstale":"2023-10-06T02:45:27.319631+0000","last_undegraded":"2023-10-06T02:45:27.319631+0000","last_fullsized":"2023-10-06T02:45:27.319631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.807177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336407+0000","last_change":"2023-10-06T02:45:23.314057+0000","last_active":"2023-10-06T02:45:27.336407+0000","last_peered":"2023-10-06T02:45:27.336407+0000","last_clean":"2023-10-06T02:45:27.336407+0000","last_became_active":"2023-10-06T02:45:23.313762+0000","last_became_peered":"2023-10-06T02:45:23.313762+0000","last_unstale":"2023-10-06T02:45:27.336407+0000","last_undegraded":"2023-10-06T02:45:27.336407+0000","last_fullsized":"2023-10-06T02:45:27.336407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:54.392205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341622+0000","last_change":"2023-10-06T02:45:23.318996+0000","last_active":"2023-10-06T02:45:27.341622+0000","last_peered":"2023-10-06T02:45:27.341622+0000","last_clean":"2023-10-06T02:45:27.341622+0000","last_became_active":"2023-10-06T02:45:23.318710+0000","last_became_peered":"2023-10-06T02:45:23.318710+0000","last_unstale":"2023-10-06T02:45:27.341622+0000","last_undegraded":"2023-10-06T02:45:27.341622+0000","last_fullsized":"2023-10-06T02:45:27.341622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:04.789619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334795+0000","last_change":"2023-10-06T02:45:23.312737+0000","last_active":"2023-10-06T02:45:27.334795+0000","last_peered":"2023-10-06T02:45:27.334795+0000","last_clean":"2023-10-06T02:45:27.334795+0000","last_became_active":"2023-10-06T02:45:23.312446+0000","last_became_peered":"2023-10-06T02:45:23.312446+0000","last_unstale":"2023-10-06T02:45:27.334795+0000","last_undegraded":"2023-10-06T02:45:27.334795+0000","last_fullsized":"2023-10-06T02:45:27.334795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340404+0000","last_change":"2023-10-06T02:45:23.318887+0000","last_active":"2023-10-06T02:45:27.340404+0000","last_peered":"2023-10-06T02:45:27.340404+0000","last_clean":"2023-10-06T02:45:27.340404+0000","last_became_active":"2023-10-06T02:45:23.318573+0000","last_became_peered":"2023-10-06T02:45:23.318573+0000","last_unstale":"2023-10-06T02:45:27.340404+0000","last_undegraded":"2023-10-06T02:45:27.340404+0000","last_fullsized":"2023-10-06T02:45:27.340404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331986+0000","last_change":"2023-10-06T02:45:23.326294+0000","last_active":"2023-10-06T02:45:27.331986+0000","last_peered":"2023-10-06T02:45:27.331986+0000","last_clean":"2023-10-06T02:45:27.331986+0000","last_became_active":"2023-10-06T02:45:23.325935+0000","last_became_peered":"2023-10-06T02:45:23.325935+0000","last_unstale":"2023-10-06T02:45:27.331986+0000","last_undegraded":"2023-10-06T02:45:27.331986+0000","last_fullsized":"2023-10-06T02:45:27.331986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336417+0000","last_change":"2023-10-06T02:45:23.316621+0000","last_active":"2023-10-06T02:45:27.336417+0000","last_peered":"2023-10-06T02:45:27.336417+0000","last_clean":"2023-10-06T02:45:27.336417+0000","last_became_active":"2023-10-06T02:45:23.316283+0000","last_became_peered":"2023-10-06T02:45:23.316283+0000","last_unstale":"2023-10-06T02:45:27.336417+0000","last_undegraded":"2023-10-06T02:45:27.336417+0000","last_fullsized":"2023-10-06T02:45:27.336417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:21.187543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335121+0000","last_change":"2023-10-06T02:45:23.318488+0000","last_active":"2023-10-06T02:45:27.335121+0000","last_peered":"2023-10-06T02:45:27.335121+0000","last_clean":"2023-10-06T02:45:27.335121+0000","last_became_active":"2023-10-06T02:45:23.318144+0000","last_became_peered":"2023-10-06T02:45:23.318144+0000","last_unstale":"2023-10-06T02:45:27.335121+0000","last_undegraded":"2023-10-06T02:45:27.335121+0000","last_fullsized":"2023-10-06T02:45:27.335121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325094+0000","last_change":"2023-10-06T02:45:23.316019+0000","last_active":"2023-10-06T02:45:27.325094+0000","last_peered":"2023-10-06T02:45:27.325094+0000","last_clean":"2023-10-06T02:45:27.325094+0000","last_became_active":"2023-10-06T02:45:23.315706+0000","last_became_peered":"2023-10-06T02:45:23.315706+0000","last_unstale":"2023-10-06T02:45:27.325094+0000","last_undegraded":"2023-10-06T02:45:27.325094+0000","last_fullsized":"2023-10-06T02:45:27.325094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.498422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336118+0000","last_change":"2023-10-06T02:45:23.313938+0000","last_active":"2023-10-06T02:45:27.336118+0000","last_peered":"2023-10-06T02:45:27.336118+0000","last_clean":"2023-10-06T02:45:27.336118+0000","last_became_active":"2023-10-06T02:45:23.313340+0000","last_became_peered":"2023-10-06T02:45:23.313340+0000","last_unstale":"2023-10-06T02:45:27.336118+0000","last_undegraded":"2023-10-06T02:45:27.336118+0000","last_fullsized":"2023-10-06T02:45:27.336118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:53.828654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341529+0000","last_change":"2023-10-06T02:45:23.321544+0000","last_active":"2023-10-06T02:45:27.341529+0000","last_peered":"2023-10-06T02:45:27.341529+0000","last_clean":"2023-10-06T02:45:27.341529+0000","last_became_active":"2023-10-06T02:45:23.321216+0000","last_became_peered":"2023-10-06T02:45:23.321216+0000","last_unstale":"2023-10-06T02:45:27.341529+0000","last_undegraded":"2023-10-06T02:45:27.341529+0000","last_fullsized":"2023-10-06T02:45:27.341529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320111+0000","last_change":"2023-10-06T02:45:23.309006+0000","last_active":"2023-10-06T02:45:27.320111+0000","last_peered":"2023-10-06T02:45:27.320111+0000","last_clean":"2023-10-06T02:45:27.320111+0000","last_became_active":"2023-10-06T02:45:23.308617+0000","last_became_peered":"2023-10-06T02:45:23.308617+0000","last_unstale":"2023-10-06T02:45:27.320111+0000","last_undegraded":"2023-10-06T02:45:27.320111+0000","last_fullsized":"2023-10-06T02:45:27.320111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:35.295610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":131211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":315392,"data_stored":127531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":718363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:45:46.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:45:46.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42c2c6700 1 Processor -- start 2023-10-06T02:45:46.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42c2c6700 1 -- start start 2023-10-06T02:45:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42c2c6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4240cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42c2c6700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240cb5b0 con 0x7fc424158e70 2023-10-06T02:45:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42c2c6700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc424159240 con 0x7fc4240ccc40 2023-10-06T02:45:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42c2c6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42415a410 con 0x7fc424159fa0 2023-10-06T02:45:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42a062700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc424158e70 0x7fc4240cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44822/0 (socket says 172.21.15.33:44822) 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc42a062700 1 -- 172.21.15.33:0/3649099781 learned_addr learned my addr 172.21.15.33:0/3649099781 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429861700 1 --2- 172.21.15.33:0/3649099781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4240cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429060700 1 -- 172.21.15.33:0/3649099781 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583669054 0 0) 0x7fc4240cb5b0 con 0x7fc424158e70 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429060700 1 -- 172.21.15.33:0/3649099781 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc40c003570 con 0x7fc424158e70 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429861700 1 -- 172.21.15.33:0/3649099781 >> v1:172.21.15.167:6789/0 conn(0x7fc4240ccc40 legacy=0x7fc4240ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429060700 1 -- 172.21.15.33:0/3649099781 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071300055 0 0) 0x7fc424159240 con 0x7fc4240ccc40 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429861700 1 -- 172.21.15.33:0/3649099781 >> v1:172.21.15.33:6789/0 conn(0x7fc424158e70 legacy=0x7fc4240cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429861700 1 -- 172.21.15.33:0/3649099781 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240cd0c0 con 0x7fc424159fa0 2023-10-06T02:45:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429861700 1 --2- 172.21.15.33:0/3649099781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4240cc700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc420009e90 tx=0x7fc420009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b8c49d4a062768 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429060700 1 -- 172.21.15.33:0/3649099781 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000e070 con 0x7fc424159fa0 2023-10-06T02:45:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429060700 1 -- 172.21.15.33:0/3649099781 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000bc30 con 0x7fc424159fa0 2023-10-06T02:45:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.609+0000 7fc429060700 1 -- 172.21.15.33:0/3649099781 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420012430 con 0x7fc424159fa0 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3649099781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 msgr2=0x7fc4240cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3649099781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4240cc700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc420009e90 tx=0x7fc420009780 comp rx=0 tx=0).stop 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3649099781 shutdown_connections 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3649099781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4240cc700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3649099781 >> 172.21.15.33:0/3649099781 conn(0x7fc424067a80 msgr2=0x7fc424068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3649099781 shutdown_connections 2023-10-06T02:45:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3649099781 wait complete. 2023-10-06T02:45:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 Processor -- start 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- start start 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 0x7fc424153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424158e70 0x7fc42415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4241573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240c8810 con 0x7fc4240ccc40 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc424154fe0 con 0x7fc424158e70 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42c2c6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240c7c00 con 0x7fc424159fa0 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42a062700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 0x7fc424153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc429861700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424158e70 0x7fc42415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42a062700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 0x7fc424153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54938/0 (socket says 172.21.15.33:54938) 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc429861700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424158e70 0x7fc42415a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43966/0 (socket says 172.21.15.33:43966) 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42a062700 1 -- 172.21.15.33:0/3172937646 learned_addr learned my addr 172.21.15.33:0/3172937646 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.613+0000 7fc42a863700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4241573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42a062700 1 -- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 msgr2=0x7fc4241573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42a062700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4241573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42a062700 1 -- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424158e70 msgr2=0x7fc42415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42a062700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424158e70 0x7fc42415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42a062700 1 -- 172.21.15.33:0/3172937646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420009a20 con 0x7fc4240ccc40 2023-10-06T02:45:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42a062700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 0x7fc424153c60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c580 tx=0x7fc414007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414005b30 con 0x7fc4240ccc40 2023-10-06T02:45:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414005c90 con 0x7fc4240ccc40 2023-10-06T02:45:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4240c7ee0 con 0x7fc4240ccc40 2023-10-06T02:45:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414007320 con 0x7fc4240ccc40 2023-10-06T02:45:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240c84a0 con 0x7fc4240ccc40 2023-10-06T02:45:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.617+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4240710b0 con 0x7fc4240ccc40 2023-10-06T02:45:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.621+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc414007690 con 0x7fc4240ccc40 2023-10-06T02:45:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.621+0000 7fc41a7fc700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc410060550 0x7fc410062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.621+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc41407c010 con 0x7fc4240ccc40 2023-10-06T02:45:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.621+0000 7fc429861700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc410060550 0x7fc410062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.621+0000 7fc429861700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc410060550 0x7fc410062a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099f0 tx=0x7fc420019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.625+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc41404c1b0 con 0x7fc4240ccc40 2023-10-06T02:45:46.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.781+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4240cd5d0 con 0x7fc410060550 2023-10-06T02:45:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.801+0000 7fc41a7fc700 1 -- 172.21.15.33:0/3172937646 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367251 (secure 0 0 0) 0x7fc4240cd5d0 con 0x7fc410060550 2023-10-06T02:45:46.812 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc410060550 msgr2=0x7fc410062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc410060550 0x7fc410062a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099f0 tx=0x7fc420019000 comp rx=0 tx=0).stop 2023-10-06T02:45:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 msgr2=0x7fc424153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 0x7fc424153c60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c580 tx=0x7fc414007df0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 shutdown_connections 2023-10-06T02:45:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc410060550 0x7fc410062a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc424159fa0 0x7fc4241573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424158e70 0x7fc42415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 --2- 172.21.15.33:0/3172937646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4240ccc40 0x7fc424153c60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 >> 172.21.15.33:0/3172937646 conn(0x7fc424067a80 msgr2=0x7fc4240c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 shutdown_connections 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.809+0000 7fc42c2c6700 1 -- 172.21.15.33:0/3172937646 wait complete. 2023-10-06T02:45:46.816 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:45:46.829 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-10-06T02:45:46.567174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594852,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381960,"kb_used_data":3656,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592760,"statfs":{"total":773094113280,"available":770654986240,"internally_reserved":0,"allocated":3743744,"data_stored":2209232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2178,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005485"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.352797+0000","last_change":"2023-10-06T02:45:27.343811+0000","last_active":"2023-10-06T02:45:27.352797+0000","last_peered":"2023-10-06T02:45:27.352797+0000","last_clean":"2023-10-06T02:45:27.352797+0000","last_became_active":"2023-10-06T02:45:27.343175+0000","last_became_peered":"2023-10-06T02:45:27.343175+0000","last_unstale":"2023-10-06T02:45:27.352797+0000","last_undegraded":"2023-10-06T02:45:27.352797+0000","last_fullsized":"2023-10-06T02:45:27.352797+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334994+0000","last_change":"2023-10-06T02:45:22.801080+0000","last_active":"2023-10-06T02:45:27.334994+0000","last_peered":"2023-10-06T02:45:27.334994+0000","last_clean":"2023-10-06T02:45:27.334994+0000","last_became_active":"2023-10-06T02:45:22.800763+0000","last_became_peered":"2023-10-06T02:45:22.800763+0000","last_unstale":"2023-10-06T02:45:27.334994+0000","last_undegraded":"2023-10-06T02:45:27.334994+0000","last_fullsized":"2023-10-06T02:45:27.334994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333783+0000","last_change":"2023-10-06T02:45:23.305500+0000","last_active":"2023-10-06T02:45:27.333783+0000","last_peered":"2023-10-06T02:45:27.333783+0000","last_clean":"2023-10-06T02:45:27.333783+0000","last_became_active":"2023-10-06T02:45:23.305199+0000","last_became_peered":"2023-10-06T02:45:23.305199+0000","last_unstale":"2023-10-06T02:45:27.333783+0000","last_undegraded":"2023-10-06T02:45:27.333783+0000","last_fullsized":"2023-10-06T02:45:27.333783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351635+0000","last_change":"2023-10-06T02:45:27.345953+0000","last_active":"2023-10-06T02:45:27.351635+0000","last_peered":"2023-10-06T02:45:27.351635+0000","last_clean":"2023-10-06T02:45:27.351635+0000","last_became_active":"2023-10-06T02:45:27.345526+0000","last_became_peered":"2023-10-06T02:45:27.345526+0000","last_unstale":"2023-10-06T02:45:27.351635+0000","last_undegraded":"2023-10-06T02:45:27.351635+0000","last_fullsized":"2023-10-06T02:45:27.351635+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:37.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332907+0000","last_change":"2023-10-06T02:45:22.805277+0000","last_active":"2023-10-06T02:45:27.332907+0000","last_peered":"2023-10-06T02:45:27.332907+0000","last_clean":"2023-10-06T02:45:27.332907+0000","last_became_active":"2023-10-06T02:45:22.804802+0000","last_became_peered":"2023-10-06T02:45:22.804802+0000","last_unstale":"2023-10-06T02:45:27.332907+0000","last_undegraded":"2023-10-06T02:45:27.332907+0000","last_fullsized":"2023-10-06T02:45:27.332907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:37.831567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330139+0000","last_change":"2023-10-06T02:45:23.306937+0000","last_active":"2023-10-06T02:45:27.330139+0000","last_peered":"2023-10-06T02:45:27.330139+0000","last_clean":"2023-10-06T02:45:27.330139+0000","last_became_active":"2023-10-06T02:45:23.306641+0000","last_became_peered":"2023-10-06T02:45:23.306641+0000","last_unstale":"2023-10-06T02:45:27.330139+0000","last_undegraded":"2023-10-06T02:45:27.330139+0000","last_fullsized":"2023-10-06T02:45:27.330139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:23.162123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356992+0000","last_change":"2023-10-06T02:45:27.353931+0000","last_active":"2023-10-06T02:45:27.356992+0000","last_peered":"2023-10-06T02:45:27.356992+0000","last_clean":"2023-10-06T02:45:27.356992+0000","last_became_active":"2023-10-06T02:45:27.353488+0000","last_became_peered":"2023-10-06T02:45:27.353488+0000","last_unstale":"2023-10-06T02:45:27.356992+0000","last_undegraded":"2023-10-06T02:45:27.356992+0000","last_fullsized":"2023-10-06T02:45:27.356992+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341505+0000","last_change":"2023-10-06T02:45:22.803357+0000","last_active":"2023-10-06T02:45:27.341505+0000","last_peered":"2023-10-06T02:45:27.341505+0000","last_clean":"2023-10-06T02:45:27.341505+0000","last_became_active":"2023-10-06T02:45:22.800673+0000","last_became_peered":"2023-10-06T02:45:22.800673+0000","last_unstale":"2023-10-06T02:45:27.341505+0000","last_undegraded":"2023-10-06T02:45:27.341505+0000","last_fullsized":"2023-10-06T02:45:27.341505+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336470+0000","last_change":"2023-10-06T02:45:23.310269+0000","last_active":"2023-10-06T02:45:27.336470+0000","last_peered":"2023-10-06T02:45:27.336470+0000","last_clean":"2023-10-06T02:45:27.336470+0000","last_became_active":"2023-10-06T02:45:23.309927+0000","last_became_peered":"2023-10-06T02:45:23.309927+0000","last_unstale":"2023-10-06T02:45:27.336470+0000","last_undegraded":"2023-10-06T02:45:27.336470+0000","last_fullsized":"2023-10-06T02:45:27.336470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351290+0000","last_change":"2023-10-06T02:45:27.348258+0000","last_active":"2023-10-06T02:45:27.351290+0000","last_peered":"2023-10-06T02:45:27.351290+0000","last_clean":"2023-10-06T02:45:27.351290+0000","last_became_active":"2023-10-06T02:45:27.347848+0000","last_became_peered":"2023-10-06T02:45:27.347848+0000","last_unstale":"2023-10-06T02:45:27.351290+0000","last_undegraded":"2023-10-06T02:45:27.351290+0000","last_fullsized":"2023-10-06T02:45:27.351290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340794+0000","last_change":"2023-10-06T02:45:22.800086+0000","last_active":"2023-10-06T02:45:27.340794+0000","last_peered":"2023-10-06T02:45:27.340794+0000","last_clean":"2023-10-06T02:45:27.340794+0000","last_became_active":"2023-10-06T02:45:22.799739+0000","last_became_peered":"2023-10-06T02:45:22.799739+0000","last_unstale":"2023-10-06T02:45:27.340794+0000","last_undegraded":"2023-10-06T02:45:27.340794+0000","last_fullsized":"2023-10-06T02:45:27.340794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332497+0000","last_change":"2023-10-06T02:45:23.312373+0000","last_active":"2023-10-06T02:45:27.332497+0000","last_peered":"2023-10-06T02:45:27.332497+0000","last_clean":"2023-10-06T02:45:27.332497+0000","last_became_active":"2023-10-06T02:45:23.311513+0000","last_became_peered":"2023-10-06T02:45:23.311513+0000","last_unstale":"2023-10-06T02:45:27.332497+0000","last_undegraded":"2023-10-06T02:45:27.332497+0000","last_fullsized":"2023-10-06T02:45:27.332497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:10.715925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360984+0000","last_change":"2023-10-06T02:45:27.347884+0000","last_active":"2023-10-06T02:45:27.360984+0000","last_peered":"2023-10-06T02:45:27.360984+0000","last_clean":"2023-10-06T02:45:27.360984+0000","last_became_active":"2023-10-06T02:45:27.347285+0000","last_became_peered":"2023-10-06T02:45:27.347285+0000","last_unstale":"2023-10-06T02:45:27.360984+0000","last_undegraded":"2023-10-06T02:45:27.360984+0000","last_fullsized":"2023-10-06T02:45:27.360984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:52.629371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335221+0000","last_change":"2023-10-06T02:45:22.804322+0000","last_active":"2023-10-06T02:45:27.335221+0000","last_peered":"2023-10-06T02:45:27.335221+0000","last_clean":"2023-10-06T02:45:27.335221+0000","last_became_active":"2023-10-06T02:45:22.802485+0000","last_became_peered":"2023-10-06T02:45:22.802485+0000","last_unstale":"2023-10-06T02:45:27.335221+0000","last_undegraded":"2023-10-06T02:45:27.335221+0000","last_fullsized":"2023-10-06T02:45:27.335221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320001+0000","last_change":"2023-10-06T02:45:23.311929+0000","last_active":"2023-10-06T02:45:27.320001+0000","last_peered":"2023-10-06T02:45:27.320001+0000","last_clean":"2023-10-06T02:45:27.320001+0000","last_became_active":"2023-10-06T02:45:23.311209+0000","last_became_peered":"2023-10-06T02:45:23.311209+0000","last_unstale":"2023-10-06T02:45:27.320001+0000","last_undegraded":"2023-10-06T02:45:27.320001+0000","last_fullsized":"2023-10-06T02:45:27.320001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:39.089521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354213+0000","last_change":"2023-10-06T02:45:27.347958+0000","last_active":"2023-10-06T02:45:27.354213+0000","last_peered":"2023-10-06T02:45:27.354213+0000","last_clean":"2023-10-06T02:45:27.354213+0000","last_became_active":"2023-10-06T02:45:27.347490+0000","last_became_peered":"2023-10-06T02:45:27.347490+0000","last_unstale":"2023-10-06T02:45:27.354213+0000","last_undegraded":"2023-10-06T02:45:27.354213+0000","last_fullsized":"2023-10-06T02:45:27.354213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:40.393467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336567+0000","last_change":"2023-10-06T02:45:22.799339+0000","last_active":"2023-10-06T02:45:27.336567+0000","last_peered":"2023-10-06T02:45:27.336567+0000","last_clean":"2023-10-06T02:45:27.336567+0000","last_became_active":"2023-10-06T02:45:22.798982+0000","last_became_peered":"2023-10-06T02:45:22.798982+0000","last_unstale":"2023-10-06T02:45:27.336567+0000","last_undegraded":"2023-10-06T02:45:27.336567+0000","last_fullsized":"2023-10-06T02:45:27.336567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:41.206862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330449+0000","last_change":"2023-10-06T02:45:23.312269+0000","last_active":"2023-10-06T02:45:27.330449+0000","last_peered":"2023-10-06T02:45:27.330449+0000","last_clean":"2023-10-06T02:45:27.330449+0000","last_became_active":"2023-10-06T02:45:23.311975+0000","last_became_peered":"2023-10-06T02:45:23.311975+0000","last_unstale":"2023-10-06T02:45:27.330449+0000","last_undegraded":"2023-10-06T02:45:27.330449+0000","last_fullsized":"2023-10-06T02:45:27.330449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:27.851854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354212+0000","last_change":"2023-10-06T02:45:27.347553+0000","last_active":"2023-10-06T02:45:27.354212+0000","last_peered":"2023-10-06T02:45:27.354212+0000","last_clean":"2023-10-06T02:45:27.354212+0000","last_became_active":"2023-10-06T02:45:27.347165+0000","last_became_peered":"2023-10-06T02:45:27.347165+0000","last_unstale":"2023-10-06T02:45:27.354212+0000","last_undegraded":"2023-10-06T02:45:27.354212+0000","last_fullsized":"2023-10-06T02:45:27.354212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:38.681094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:22.808457+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:22.808145+0000","last_became_peered":"2023-10-06T02:45:22.808145+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334405+0000","last_change":"2023-10-06T02:45:23.323552+0000","last_active":"2023-10-06T02:45:27.334405+0000","last_peered":"2023-10-06T02:45:27.334405+0000","last_clean":"2023-10-06T02:45:27.334405+0000","last_became_active":"2023-10-06T02:45:23.323241+0000","last_became_peered":"2023-10-06T02:45:23.323241+0000","last_unstale":"2023-10-06T02:45:27.334405+0000","last_undegraded":"2023-10-06T02:45:27.334405+0000","last_fullsized":"2023-10-06T02:45:27.334405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:29.825889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353329+0000","last_change":"2023-10-06T02:45:27.349343+0000","last_active":"2023-10-06T02:45:27.353329+0000","last_peered":"2023-10-06T02:45:27.353329+0000","last_clean":"2023-10-06T02:45:27.353329+0000","last_became_active":"2023-10-06T02:45:27.348859+0000","last_became_peered":"2023-10-06T02:45:27.348859+0000","last_unstale":"2023-10-06T02:45:27.353329+0000","last_undegraded":"2023-10-06T02:45:27.353329+0000","last_fullsized":"2023-10-06T02:45:27.353329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333023+0000","last_change":"2023-10-06T02:45:22.811866+0000","last_active":"2023-10-06T02:45:27.333023+0000","last_peered":"2023-10-06T02:45:27.333023+0000","last_clean":"2023-10-06T02:45:27.333023+0000","last_became_active":"2023-10-06T02:45:22.811334+0000","last_became_peered":"2023-10-06T02:45:22.811334+0000","last_unstale":"2023-10-06T02:45:27.333023+0000","last_undegraded":"2023-10-06T02:45:27.333023+0000","last_fullsized":"2023-10-06T02:45:27.333023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325429+0000","last_change":"2023-10-06T02:45:23.320977+0000","last_active":"2023-10-06T02:45:27.325429+0000","last_peered":"2023-10-06T02:45:27.325429+0000","last_clean":"2023-10-06T02:45:27.325429+0000","last_became_active":"2023-10-06T02:45:23.320677+0000","last_became_peered":"2023-10-06T02:45:23.320677+0000","last_unstale":"2023-10-06T02:45:27.325429+0000","last_undegraded":"2023-10-06T02:45:27.325429+0000","last_fullsized":"2023-10-06T02:45:27.325429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:18.520846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332892+0000","last_change":"2023-10-06T02:45:23.312712+0000","last_active":"2023-10-06T02:45:27.332892+0000","last_peered":"2023-10-06T02:45:27.332892+0000","last_clean":"2023-10-06T02:45:27.332892+0000","last_became_active":"2023-10-06T02:45:23.312164+0000","last_became_peered":"2023-10-06T02:45:23.312164+0000","last_unstale":"2023-10-06T02:45:27.332892+0000","last_undegraded":"2023-10-06T02:45:27.332892+0000","last_fullsized":"2023-10-06T02:45:27.332892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:22.186960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335089+0000","last_change":"2023-10-06T02:45:22.800781+0000","last_active":"2023-10-06T02:45:27.335089+0000","last_peered":"2023-10-06T02:45:27.335089+0000","last_clean":"2023-10-06T02:45:27.335089+0000","last_became_active":"2023-10-06T02:45:22.800484+0000","last_became_peered":"2023-10-06T02:45:22.800484+0000","last_unstale":"2023-10-06T02:45:27.335089+0000","last_undegraded":"2023-10-06T02:45:27.335089+0000","last_fullsized":"2023-10-06T02:45:27.335089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354176+0000","last_change":"2023-10-06T02:45:27.348699+0000","last_active":"2023-10-06T02:45:27.354176+0000","last_peered":"2023-10-06T02:45:27.354176+0000","last_clean":"2023-10-06T02:45:27.354176+0000","last_became_active":"2023-10-06T02:45:27.348111+0000","last_became_peered":"2023-10-06T02:45:27.348111+0000","last_unstale":"2023-10-06T02:45:27.354176+0000","last_undegraded":"2023-10-06T02:45:27.354176+0000","last_fullsized":"2023-10-06T02:45:27.354176+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:26.404420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330041+0000","last_change":"2023-10-06T02:45:23.323471+0000","last_active":"2023-10-06T02:45:27.330041+0000","last_peered":"2023-10-06T02:45:27.330041+0000","last_clean":"2023-10-06T02:45:27.330041+0000","last_became_active":"2023-10-06T02:45:23.323203+0000","last_became_peered":"2023-10-06T02:45:23.323203+0000","last_unstale":"2023-10-06T02:45:27.330041+0000","last_undegraded":"2023-10-06T02:45:27.330041+0000","last_fullsized":"2023-10-06T02:45:27.330041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:59.446720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341624+0000","last_change":"2023-10-06T02:45:22.804313+0000","last_active":"2023-10-06T02:45:27.341624+0000","last_peered":"2023-10-06T02:45:27.341624+0000","last_clean":"2023-10-06T02:45:27.341624+0000","last_became_active":"2023-10-06T02:45:22.804008+0000","last_became_peered":"2023-10-06T02:45:22.804008+0000","last_unstale":"2023-10-06T02:45:27.341624+0000","last_undegraded":"2023-10-06T02:45:27.341624+0000","last_fullsized":"2023-10-06T02:45:27.341624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356829+0000","last_change":"2023-10-06T02:45:27.355277+0000","last_active":"2023-10-06T02:45:27.356829+0000","last_peered":"2023-10-06T02:45:27.356829+0000","last_clean":"2023-10-06T02:45:27.356829+0000","last_became_active":"2023-10-06T02:45:27.354885+0000","last_became_peered":"2023-10-06T02:45:27.354885+0000","last_unstale":"2023-10-06T02:45:27.356829+0000","last_undegraded":"2023-10-06T02:45:27.356829+0000","last_fullsized":"2023-10-06T02:45:27.356829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.722369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333001+0000","last_change":"2023-10-06T02:45:22.810372+0000","last_active":"2023-10-06T02:45:27.333001+0000","last_peered":"2023-10-06T02:45:27.333001+0000","last_clean":"2023-10-06T02:45:27.333001+0000","last_became_active":"2023-10-06T02:45:22.810117+0000","last_became_peered":"2023-10-06T02:45:22.810117+0000","last_unstale":"2023-10-06T02:45:27.333001+0000","last_undegraded":"2023-10-06T02:45:27.333001+0000","last_fullsized":"2023-10-06T02:45:27.333001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:54.333503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335943+0000","last_change":"2023-10-06T02:45:23.319583+0000","last_active":"2023-10-06T02:45:27.335943+0000","last_peered":"2023-10-06T02:45:27.335943+0000","last_clean":"2023-10-06T02:45:27.335943+0000","last_became_active":"2023-10-06T02:45:23.319266+0000","last_became_peered":"2023-10-06T02:45:23.319266+0000","last_unstale":"2023-10-06T02:45:27.335943+0000","last_undegraded":"2023-10-06T02:45:27.335943+0000","last_fullsized":"2023-10-06T02:45:27.335943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353966+0000","last_change":"2023-10-06T02:45:27.351069+0000","last_active":"2023-10-06T02:45:27.353966+0000","last_peered":"2023-10-06T02:45:27.353966+0000","last_clean":"2023-10-06T02:45:27.353966+0000","last_became_active":"2023-10-06T02:45:27.350579+0000","last_became_peered":"2023-10-06T02:45:27.350579+0000","last_unstale":"2023-10-06T02:45:27.353966+0000","last_undegraded":"2023-10-06T02:45:27.353966+0000","last_fullsized":"2023-10-06T02:45:27.353966+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:47.948820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"29'9","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:33.113646+0000","last_change":"2023-10-06T02:45:22.804024+0000","last_active":"2023-10-06T02:45:33.113646+0000","last_peered":"2023-10-06T02:45:33.113646+0000","last_clean":"2023-10-06T02:45:33.113646+0000","last_became_active":"2023-10-06T02:45:22.803679+0000","last_became_peered":"2023-10-06T02:45:22.803679+0000","last_unstale":"2023-10-06T02:45:33.113646+0000","last_undegraded":"2023-10-06T02:45:33.113646+0000","last_fullsized":"2023-10-06T02:45:33.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:12.507771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332592+0000","last_change":"2023-10-06T02:45:23.312638+0000","last_active":"2023-10-06T02:45:27.332592+0000","last_peered":"2023-10-06T02:45:27.332592+0000","last_clean":"2023-10-06T02:45:27.332592+0000","last_became_active":"2023-10-06T02:45:23.312006+0000","last_became_peered":"2023-10-06T02:45:23.312006+0000","last_unstale":"2023-10-06T02:45:27.332592+0000","last_undegraded":"2023-10-06T02:45:27.332592+0000","last_fullsized":"2023-10-06T02:45:27.332592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.358568+0000","last_change":"2023-10-06T02:45:27.345474+0000","last_active":"2023-10-06T02:45:27.358568+0000","last_peered":"2023-10-06T02:45:27.358568+0000","last_clean":"2023-10-06T02:45:27.358568+0000","last_became_active":"2023-10-06T02:45:27.345014+0000","last_became_peered":"2023-10-06T02:45:27.345014+0000","last_unstale":"2023-10-06T02:45:27.358568+0000","last_undegraded":"2023-10-06T02:45:27.358568+0000","last_fullsized":"2023-10-06T02:45:27.358568+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:20.057147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334989+0000","last_change":"2023-10-06T02:45:23.319302+0000","last_active":"2023-10-06T02:45:27.334989+0000","last_peered":"2023-10-06T02:45:27.334989+0000","last_clean":"2023-10-06T02:45:27.334989+0000","last_became_active":"2023-10-06T02:45:23.318951+0000","last_became_peered":"2023-10-06T02:45:23.318951+0000","last_unstale":"2023-10-06T02:45:27.334989+0000","last_undegraded":"2023-10-06T02:45:27.334989+0000","last_fullsized":"2023-10-06T02:45:27.334989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:34.946622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325697+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:45:27.325697+0000","last_peered":"2023-10-06T02:45:27.325697+0000","last_clean":"2023-10-06T02:45:27.325697+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:45:27.325697+0000","last_undegraded":"2023-10-06T02:45:27.325697+0000","last_fullsized":"2023-10-06T02:45:27.325697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'1","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T02:45:28.296097+0000","last_change":"2023-10-06T02:45:22.799764+0000","last_active":"2023-10-06T02:45:28.296097+0000","last_peered":"2023-10-06T02:45:28.296097+0000","last_clean":"2023-10-06T02:45:28.296097+0000","last_became_active":"2023-10-06T02:45:22.798917+0000","last_became_peered":"2023-10-06T02:45:22.798917+0000","last_unstale":"2023-10-06T02:45:28.296097+0000","last_undegraded":"2023-10-06T02:45:28.296097+0000","last_fullsized":"2023-10-06T02:45:28.296097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353993+0000","last_change":"2023-10-06T02:45:27.350164+0000","last_active":"2023-10-06T02:45:27.353993+0000","last_peered":"2023-10-06T02:45:27.353993+0000","last_clean":"2023-10-06T02:45:27.353993+0000","last_became_active":"2023-10-06T02:45:27.349731+0000","last_became_peered":"2023-10-06T02:45:27.349731+0000","last_unstale":"2023-10-06T02:45:27.353993+0000","last_undegraded":"2023-10-06T02:45:27.353993+0000","last_fullsized":"2023-10-06T02:45:27.353993+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:07.179714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332977+0000","last_change":"2023-10-06T02:45:23.324382+0000","last_active":"2023-10-06T02:45:27.332977+0000","last_peered":"2023-10-06T02:45:27.332977+0000","last_clean":"2023-10-06T02:45:27.332977+0000","last_became_active":"2023-10-06T02:45:23.324017+0000","last_became_peered":"2023-10-06T02:45:23.324017+0000","last_unstale":"2023-10-06T02:45:27.332977+0000","last_undegraded":"2023-10-06T02:45:27.332977+0000","last_fullsized":"2023-10-06T02:45:27.332977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:01.347856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340460+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:45:27.340460+0000","last_peered":"2023-10-06T02:45:27.340460+0000","last_clean":"2023-10-06T02:45:27.340460+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:45:27.340460+0000","last_undegraded":"2023-10-06T02:45:27.340460+0000","last_fullsized":"2023-10-06T02:45:27.340460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333588+0000","last_change":"2023-10-06T02:45:22.807333+0000","last_active":"2023-10-06T02:45:27.333588+0000","last_peered":"2023-10-06T02:45:27.333588+0000","last_clean":"2023-10-06T02:45:27.333588+0000","last_became_active":"2023-10-06T02:45:22.805988+0000","last_became_peered":"2023-10-06T02:45:22.805988+0000","last_unstale":"2023-10-06T02:45:27.333588+0000","last_undegraded":"2023-10-06T02:45:27.333588+0000","last_fullsized":"2023-10-06T02:45:27.333588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:41.870262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353464+0000","last_change":"2023-10-06T02:45:27.347859+0000","last_active":"2023-10-06T02:45:27.353464+0000","last_peered":"2023-10-06T02:45:27.353464+0000","last_clean":"2023-10-06T02:45:27.353464+0000","last_became_active":"2023-10-06T02:45:27.347418+0000","last_became_peered":"2023-10-06T02:45:27.347418+0000","last_unstale":"2023-10-06T02:45:27.353464+0000","last_undegraded":"2023-10-06T02:45:27.353464+0000","last_fullsized":"2023-10-06T02:45:27.353464+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:17.541971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334956+0000","last_change":"2023-10-06T02:45:23.320296+0000","last_active":"2023-10-06T02:45:27.334956+0000","last_peered":"2023-10-06T02:45:27.334956+0000","last_clean":"2023-10-06T02:45:27.334956+0000","last_became_active":"2023-10-06T02:45:23.319983+0000","last_became_peered":"2023-10-06T02:45:23.319983+0000","last_unstale":"2023-10-06T02:45:27.334956+0000","last_undegraded":"2023-10-06T02:45:27.334956+0000","last_fullsized":"2023-10-06T02:45:27.334956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333252+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:45:27.333252+0000","last_peered":"2023-10-06T02:45:27.333252+0000","last_clean":"2023-10-06T02:45:27.333252+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:45:27.333252+0000","last_undegraded":"2023-10-06T02:45:27.333252+0000","last_fullsized":"2023-10-06T02:45:27.333252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336685+0000","last_change":"2023-10-06T02:45:22.804019+0000","last_active":"2023-10-06T02:45:27.336685+0000","last_peered":"2023-10-06T02:45:27.336685+0000","last_clean":"2023-10-06T02:45:27.336685+0000","last_became_active":"2023-10-06T02:45:22.803524+0000","last_became_peered":"2023-10-06T02:45:22.803524+0000","last_unstale":"2023-10-06T02:45:27.336685+0000","last_undegraded":"2023-10-06T02:45:27.336685+0000","last_fullsized":"2023-10-06T02:45:27.336685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:28.802655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353592+0000","last_change":"2023-10-06T02:45:27.344027+0000","last_active":"2023-10-06T02:45:27.353592+0000","last_peered":"2023-10-06T02:45:27.353592+0000","last_clean":"2023-10-06T02:45:27.353592+0000","last_became_active":"2023-10-06T02:45:27.343450+0000","last_became_peered":"2023-10-06T02:45:27.343450+0000","last_unstale":"2023-10-06T02:45:27.353592+0000","last_undegraded":"2023-10-06T02:45:27.353592+0000","last_fullsized":"2023-10-06T02:45:27.353592+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:42.951337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336038+0000","last_change":"2023-10-06T02:45:23.319757+0000","last_active":"2023-10-06T02:45:27.336038+0000","last_peered":"2023-10-06T02:45:27.336038+0000","last_clean":"2023-10-06T02:45:27.336038+0000","last_became_active":"2023-10-06T02:45:23.319356+0000","last_became_peered":"2023-10-06T02:45:23.319356+0000","last_unstale":"2023-10-06T02:45:27.336038+0000","last_undegraded":"2023-10-06T02:45:27.336038+0000","last_fullsized":"2023-10-06T02:45:27.336038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337310+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:45:27.337310+0000","last_peered":"2023-10-06T02:45:27.337310+0000","last_clean":"2023-10-06T02:45:27.337310+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:45:27.337310+0000","last_undegraded":"2023-10-06T02:45:27.337310+0000","last_fullsized":"2023-10-06T02:45:27.337310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341535+0000","last_change":"2023-10-06T02:45:22.808675+0000","last_active":"2023-10-06T02:45:27.341535+0000","last_peered":"2023-10-06T02:45:27.341535+0000","last_clean":"2023-10-06T02:45:27.341535+0000","last_became_active":"2023-10-06T02:45:22.807693+0000","last_became_peered":"2023-10-06T02:45:22.807693+0000","last_unstale":"2023-10-06T02:45:27.341535+0000","last_undegraded":"2023-10-06T02:45:27.341535+0000","last_fullsized":"2023-10-06T02:45:27.341535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356445+0000","last_change":"2023-10-06T02:45:27.352852+0000","last_active":"2023-10-06T02:45:27.356445+0000","last_peered":"2023-10-06T02:45:27.356445+0000","last_clean":"2023-10-06T02:45:27.356445+0000","last_became_active":"2023-10-06T02:45:27.352303+0000","last_became_peered":"2023-10-06T02:45:27.352303+0000","last_unstale":"2023-10-06T02:45:27.356445+0000","last_undegraded":"2023-10-06T02:45:27.356445+0000","last_fullsized":"2023-10-06T02:45:27.356445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:21.854256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336597+0000","last_change":"2023-10-06T02:45:23.317570+0000","last_active":"2023-10-06T02:45:27.336597+0000","last_peered":"2023-10-06T02:45:27.336597+0000","last_clean":"2023-10-06T02:45:27.336597+0000","last_became_active":"2023-10-06T02:45:23.316419+0000","last_became_peered":"2023-10-06T02:45:23.316419+0000","last_unstale":"2023-10-06T02:45:27.336597+0000","last_undegraded":"2023-10-06T02:45:27.336597+0000","last_fullsized":"2023-10-06T02:45:27.336597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332685+0000","last_change":"2023-10-06T02:45:22.799727+0000","last_active":"2023-10-06T02:45:27.332685+0000","last_peered":"2023-10-06T02:45:27.332685+0000","last_clean":"2023-10-06T02:45:27.332685+0000","last_became_active":"2023-10-06T02:45:22.799276+0000","last_became_peered":"2023-10-06T02:45:22.799276+0000","last_unstale":"2023-10-06T02:45:27.332685+0000","last_undegraded":"2023-10-06T02:45:27.332685+0000","last_fullsized":"2023-10-06T02:45:27.332685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:23.261429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340308+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:45:27.340308+0000","last_peered":"2023-10-06T02:45:27.340308+0000","last_clean":"2023-10-06T02:45:27.340308+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:45:27.340308+0000","last_undegraded":"2023-10-06T02:45:27.340308+0000","last_fullsized":"2023-10-06T02:45:27.340308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.353669+0000","last_change":"2023-10-06T02:45:27.350898+0000","last_active":"2023-10-06T02:45:27.353669+0000","last_peered":"2023-10-06T02:45:27.353669+0000","last_clean":"2023-10-06T02:45:27.353669+0000","last_became_active":"2023-10-06T02:45:27.350489+0000","last_became_peered":"2023-10-06T02:45:27.350489+0000","last_unstale":"2023-10-06T02:45:27.353669+0000","last_undegraded":"2023-10-06T02:45:27.353669+0000","last_fullsized":"2023-10-06T02:45:27.353669+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340516+0000","last_change":"2023-10-06T02:45:23.320581+0000","last_active":"2023-10-06T02:45:27.340516+0000","last_peered":"2023-10-06T02:45:27.340516+0000","last_clean":"2023-10-06T02:45:27.340516+0000","last_became_active":"2023-10-06T02:45:23.320253+0000","last_became_peered":"2023-10-06T02:45:23.320253+0000","last_unstale":"2023-10-06T02:45:27.340516+0000","last_undegraded":"2023-10-06T02:45:27.340516+0000","last_fullsized":"2023-10-06T02:45:27.340516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:15.466869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334594+0000","last_change":"2023-10-06T02:45:22.803922+0000","last_active":"2023-10-06T02:45:27.334594+0000","last_peered":"2023-10-06T02:45:27.334594+0000","last_clean":"2023-10-06T02:45:27.334594+0000","last_became_active":"2023-10-06T02:45:22.803487+0000","last_became_peered":"2023-10-06T02:45:22.803487+0000","last_unstale":"2023-10-06T02:45:27.334594+0000","last_undegraded":"2023-10-06T02:45:27.334594+0000","last_fullsized":"2023-10-06T02:45:27.334594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319371+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:45:27.319371+0000","last_peered":"2023-10-06T02:45:27.319371+0000","last_clean":"2023-10-06T02:45:27.319371+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:45:27.319371+0000","last_undegraded":"2023-10-06T02:45:27.319371+0000","last_fullsized":"2023-10-06T02:45:27.319371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350330+0000","last_change":"2023-10-06T02:45:27.341548+0000","last_active":"2023-10-06T02:45:27.350330+0000","last_peered":"2023-10-06T02:45:27.350330+0000","last_clean":"2023-10-06T02:45:27.350330+0000","last_became_active":"2023-10-06T02:45:27.341116+0000","last_became_peered":"2023-10-06T02:45:27.341116+0000","last_unstale":"2023-10-06T02:45:27.350330+0000","last_undegraded":"2023-10-06T02:45:27.350330+0000","last_fullsized":"2023-10-06T02:45:27.350330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:12.366117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334501+0000","last_change":"2023-10-06T02:45:23.313236+0000","last_active":"2023-10-06T02:45:27.334501+0000","last_peered":"2023-10-06T02:45:27.334501+0000","last_clean":"2023-10-06T02:45:27.334501+0000","last_became_active":"2023-10-06T02:45:23.312943+0000","last_became_peered":"2023-10-06T02:45:23.312943+0000","last_unstale":"2023-10-06T02:45:27.334501+0000","last_undegraded":"2023-10-06T02:45:27.334501+0000","last_fullsized":"2023-10-06T02:45:27.334501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333980+0000","last_change":"2023-10-06T02:45:22.801183+0000","last_active":"2023-10-06T02:45:27.333980+0000","last_peered":"2023-10-06T02:45:27.333980+0000","last_clean":"2023-10-06T02:45:27.333980+0000","last_became_active":"2023-10-06T02:45:22.800872+0000","last_became_peered":"2023-10-06T02:45:22.800872+0000","last_unstale":"2023-10-06T02:45:27.333980+0000","last_undegraded":"2023-10-06T02:45:27.333980+0000","last_fullsized":"2023-10-06T02:45:27.333980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331616+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:45:27.331616+0000","last_peered":"2023-10-06T02:45:27.331616+0000","last_clean":"2023-10-06T02:45:27.331616+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:45:27.331616+0000","last_undegraded":"2023-10-06T02:45:27.331616+0000","last_fullsized":"2023-10-06T02:45:27.331616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357435+0000","last_change":"2023-10-06T02:45:27.354883+0000","last_active":"2023-10-06T02:45:27.357435+0000","last_peered":"2023-10-06T02:45:27.357435+0000","last_clean":"2023-10-06T02:45:27.357435+0000","last_became_active":"2023-10-06T02:45:27.353904+0000","last_became_peered":"2023-10-06T02:45:27.353904+0000","last_unstale":"2023-10-06T02:45:27.357435+0000","last_undegraded":"2023-10-06T02:45:27.357435+0000","last_fullsized":"2023-10-06T02:45:27.357435+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:51:54.845793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325620+0000","last_change":"2023-10-06T02:45:23.311475+0000","last_active":"2023-10-06T02:45:27.325620+0000","last_peered":"2023-10-06T02:45:27.325620+0000","last_clean":"2023-10-06T02:45:27.325620+0000","last_became_active":"2023-10-06T02:45:23.311183+0000","last_became_peered":"2023-10-06T02:45:23.311183+0000","last_unstale":"2023-10-06T02:45:27.325620+0000","last_undegraded":"2023-10-06T02:45:27.325620+0000","last_fullsized":"2023-10-06T02:45:27.325620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:38.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331867+0000","last_change":"2023-10-06T02:45:22.800538+0000","last_active":"2023-10-06T02:45:27.331867+0000","last_peered":"2023-10-06T02:45:27.331867+0000","last_clean":"2023-10-06T02:45:27.331867+0000","last_became_active":"2023-10-06T02:45:22.800251+0000","last_became_peered":"2023-10-06T02:45:22.800251+0000","last_unstale":"2023-10-06T02:45:27.331867+0000","last_undegraded":"2023-10-06T02:45:27.331867+0000","last_fullsized":"2023-10-06T02:45:27.331867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:35.553098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340818+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:45:27.340818+0000","last_peered":"2023-10-06T02:45:27.340818+0000","last_clean":"2023-10-06T02:45:27.340818+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:45:27.340818+0000","last_undegraded":"2023-10-06T02:45:27.340818+0000","last_fullsized":"2023-10-06T02:45:27.340818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331708+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:45:27.331708+0000","last_peered":"2023-10-06T02:45:27.331708+0000","last_clean":"2023-10-06T02:45:27.331708+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:45:27.331708+0000","last_undegraded":"2023-10-06T02:45:27.331708+0000","last_fullsized":"2023-10-06T02:45:27.331708+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351388+0000","last_change":"2023-10-06T02:45:27.349035+0000","last_active":"2023-10-06T02:45:27.351388+0000","last_peered":"2023-10-06T02:45:27.351388+0000","last_clean":"2023-10-06T02:45:27.351388+0000","last_became_active":"2023-10-06T02:45:27.348608+0000","last_became_peered":"2023-10-06T02:45:27.348608+0000","last_unstale":"2023-10-06T02:45:27.351388+0000","last_undegraded":"2023-10-06T02:45:27.351388+0000","last_fullsized":"2023-10-06T02:45:27.351388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:18.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340259+0000","last_change":"2023-10-06T02:45:23.310743+0000","last_active":"2023-10-06T02:45:27.340259+0000","last_peered":"2023-10-06T02:45:27.340259+0000","last_clean":"2023-10-06T02:45:27.340259+0000","last_became_active":"2023-10-06T02:45:23.310401+0000","last_became_peered":"2023-10-06T02:45:23.310401+0000","last_unstale":"2023-10-06T02:45:27.340259+0000","last_undegraded":"2023-10-06T02:45:27.340259+0000","last_fullsized":"2023-10-06T02:45:27.340259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333057+0000","last_change":"2023-10-06T02:45:22.807407+0000","last_active":"2023-10-06T02:45:27.333057+0000","last_peered":"2023-10-06T02:45:27.333057+0000","last_clean":"2023-10-06T02:45:27.333057+0000","last_became_active":"2023-10-06T02:45:22.804974+0000","last_became_peered":"2023-10-06T02:45:22.804974+0000","last_unstale":"2023-10-06T02:45:27.333057+0000","last_undegraded":"2023-10-06T02:45:27.333057+0000","last_fullsized":"2023-10-06T02:45:27.333057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:15.722413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.351625+0000","last_change":"2023-10-06T02:45:27.346225+0000","last_active":"2023-10-06T02:45:27.351625+0000","last_peered":"2023-10-06T02:45:27.351625+0000","last_clean":"2023-10-06T02:45:27.351625+0000","last_became_active":"2023-10-06T02:45:27.345701+0000","last_became_peered":"2023-10-06T02:45:27.345701+0000","last_unstale":"2023-10-06T02:45:27.351625+0000","last_undegraded":"2023-10-06T02:45:27.351625+0000","last_fullsized":"2023-10-06T02:45:27.351625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:56.453178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334551+0000","last_change":"2023-10-06T02:45:23.310126+0000","last_active":"2023-10-06T02:45:27.334551+0000","last_peered":"2023-10-06T02:45:27.334551+0000","last_clean":"2023-10-06T02:45:27.334551+0000","last_became_active":"2023-10-06T02:45:23.309805+0000","last_became_peered":"2023-10-06T02:45:23.309805+0000","last_unstale":"2023-10-06T02:45:27.334551+0000","last_undegraded":"2023-10-06T02:45:27.334551+0000","last_fullsized":"2023-10-06T02:45:27.334551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336853+0000","last_change":"2023-10-06T02:45:22.804564+0000","last_active":"2023-10-06T02:45:27.336853+0000","last_peered":"2023-10-06T02:45:27.336853+0000","last_clean":"2023-10-06T02:45:27.336853+0000","last_became_active":"2023-10-06T02:45:22.804286+0000","last_became_peered":"2023-10-06T02:45:22.804286+0000","last_unstale":"2023-10-06T02:45:27.336853+0000","last_undegraded":"2023-10-06T02:45:27.336853+0000","last_fullsized":"2023-10-06T02:45:27.336853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:34.117511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357099+0000","last_change":"2023-10-06T02:45:27.354769+0000","last_active":"2023-10-06T02:45:27.357099+0000","last_peered":"2023-10-06T02:45:27.357099+0000","last_clean":"2023-10-06T02:45:27.357099+0000","last_became_active":"2023-10-06T02:45:27.352768+0000","last_became_peered":"2023-10-06T02:45:27.352768+0000","last_unstale":"2023-10-06T02:45:27.357099+0000","last_undegraded":"2023-10-06T02:45:27.357099+0000","last_fullsized":"2023-10-06T02:45:27.357099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:41.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336239+0000","last_change":"2023-10-06T02:45:23.310059+0000","last_active":"2023-10-06T02:45:27.336239+0000","last_peered":"2023-10-06T02:45:27.336239+0000","last_clean":"2023-10-06T02:45:27.336239+0000","last_became_active":"2023-10-06T02:45:23.309667+0000","last_became_peered":"2023-10-06T02:45:23.309667+0000","last_unstale":"2023-10-06T02:45:27.336239+0000","last_undegraded":"2023-10-06T02:45:27.336239+0000","last_fullsized":"2023-10-06T02:45:27.336239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:28.228899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.326136+0000","last_change":"2023-10-06T02:45:22.795986+0000","last_active":"2023-10-06T02:45:27.326136+0000","last_peered":"2023-10-06T02:45:27.326136+0000","last_clean":"2023-10-06T02:45:27.326136+0000","last_became_active":"2023-10-06T02:45:22.795649+0000","last_became_peered":"2023-10-06T02:45:22.795649+0000","last_unstale":"2023-10-06T02:45:27.326136+0000","last_undegraded":"2023-10-06T02:45:27.326136+0000","last_fullsized":"2023-10-06T02:45:27.326136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:52.473292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.360933+0000","last_change":"2023-10-06T02:45:27.347788+0000","last_active":"2023-10-06T02:45:27.360933+0000","last_peered":"2023-10-06T02:45:27.360933+0000","last_clean":"2023-10-06T02:45:27.360933+0000","last_became_active":"2023-10-06T02:45:27.347192+0000","last_became_peered":"2023-10-06T02:45:27.347192+0000","last_unstale":"2023-10-06T02:45:27.360933+0000","last_undegraded":"2023-10-06T02:45:27.360933+0000","last_fullsized":"2023-10-06T02:45:27.360933+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:21.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336639+0000","last_change":"2023-10-06T02:45:23.307192+0000","last_active":"2023-10-06T02:45:27.336639+0000","last_peered":"2023-10-06T02:45:27.336639+0000","last_clean":"2023-10-06T02:45:27.336639+0000","last_became_active":"2023-10-06T02:45:23.306642+0000","last_became_peered":"2023-10-06T02:45:23.306642+0000","last_unstale":"2023-10-06T02:45:27.336639+0000","last_undegraded":"2023-10-06T02:45:27.336639+0000","last_fullsized":"2023-10-06T02:45:27.336639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:14.468949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333926+0000","last_change":"2023-10-06T02:45:22.811906+0000","last_active":"2023-10-06T02:45:27.333926+0000","last_peered":"2023-10-06T02:45:27.333926+0000","last_clean":"2023-10-06T02:45:27.333926+0000","last_became_active":"2023-10-06T02:45:22.811602+0000","last_became_peered":"2023-10-06T02:45:22.811602+0000","last_unstale":"2023-10-06T02:45:27.333926+0000","last_undegraded":"2023-10-06T02:45:27.333926+0000","last_fullsized":"2023-10-06T02:45:27.333926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.090764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.349516+0000","last_change":"2023-10-06T02:45:27.344016+0000","last_active":"2023-10-06T02:45:27.349516+0000","last_peered":"2023-10-06T02:45:27.349516+0000","last_clean":"2023-10-06T02:45:27.349516+0000","last_became_active":"2023-10-06T02:45:27.342950+0000","last_became_peered":"2023-10-06T02:45:27.342950+0000","last_unstale":"2023-10-06T02:45:27.349516+0000","last_undegraded":"2023-10-06T02:45:27.349516+0000","last_fullsized":"2023-10-06T02:45:27.349516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:25.726296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334869+0000","last_change":"2023-10-06T02:45:23.321013+0000","last_active":"2023-10-06T02:45:27.334869+0000","last_peered":"2023-10-06T02:45:27.334869+0000","last_clean":"2023-10-06T02:45:27.334869+0000","last_became_active":"2023-10-06T02:45:23.320682+0000","last_became_peered":"2023-10-06T02:45:23.320682+0000","last_unstale":"2023-10-06T02:45:27.334869+0000","last_undegraded":"2023-10-06T02:45:27.334869+0000","last_fullsized":"2023-10-06T02:45:27.334869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341800+0000","last_change":"2023-10-06T02:45:22.806416+0000","last_active":"2023-10-06T02:45:27.341800+0000","last_peered":"2023-10-06T02:45:27.341800+0000","last_clean":"2023-10-06T02:45:27.341800+0000","last_became_active":"2023-10-06T02:45:22.806087+0000","last_became_peered":"2023-10-06T02:45:22.806087+0000","last_unstale":"2023-10-06T02:45:27.341800+0000","last_undegraded":"2023-10-06T02:45:27.341800+0000","last_fullsized":"2023-10-06T02:45:27.341800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357937+0000","last_change":"2023-10-06T02:45:27.350897+0000","last_active":"2023-10-06T02:45:27.357937+0000","last_peered":"2023-10-06T02:45:27.357937+0000","last_clean":"2023-10-06T02:45:27.357937+0000","last_became_active":"2023-10-06T02:45:27.350497+0000","last_became_peered":"2023-10-06T02:45:27.350497+0000","last_unstale":"2023-10-06T02:45:27.357937+0000","last_undegraded":"2023-10-06T02:45:27.357937+0000","last_fullsized":"2023-10-06T02:45:27.357937+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:38.481448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325860+0000","last_change":"2023-10-06T02:45:23.313267+0000","last_active":"2023-10-06T02:45:27.325860+0000","last_peered":"2023-10-06T02:45:27.325860+0000","last_clean":"2023-10-06T02:45:27.325860+0000","last_became_active":"2023-10-06T02:45:23.312886+0000","last_became_peered":"2023-10-06T02:45:23.312886+0000","last_unstale":"2023-10-06T02:45:27.325860+0000","last_undegraded":"2023-10-06T02:45:27.325860+0000","last_fullsized":"2023-10-06T02:45:27.325860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:30.923831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333106+0000","last_change":"2023-10-06T02:45:22.809724+0000","last_active":"2023-10-06T02:45:27.333106+0000","last_peered":"2023-10-06T02:45:27.333106+0000","last_clean":"2023-10-06T02:45:27.333106+0000","last_became_active":"2023-10-06T02:45:22.809451+0000","last_became_peered":"2023-10-06T02:45:22.809451+0000","last_unstale":"2023-10-06T02:45:27.333106+0000","last_undegraded":"2023-10-06T02:45:27.333106+0000","last_fullsized":"2023-10-06T02:45:27.333106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:10.054721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.348579+0000","last_change":"2023-10-06T02:45:27.343402+0000","last_active":"2023-10-06T02:45:27.348579+0000","last_peered":"2023-10-06T02:45:27.348579+0000","last_clean":"2023-10-06T02:45:27.348579+0000","last_became_active":"2023-10-06T02:45:27.342892+0000","last_became_peered":"2023-10-06T02:45:27.342892+0000","last_unstale":"2023-10-06T02:45:27.348579+0000","last_undegraded":"2023-10-06T02:45:27.348579+0000","last_fullsized":"2023-10-06T02:45:27.348579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:05.116878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334800+0000","last_change":"2023-10-06T02:45:23.313977+0000","last_active":"2023-10-06T02:45:27.334800+0000","last_peered":"2023-10-06T02:45:27.334800+0000","last_clean":"2023-10-06T02:45:27.334800+0000","last_became_active":"2023-10-06T02:45:23.313656+0000","last_became_peered":"2023-10-06T02:45:23.313656+0000","last_unstale":"2023-10-06T02:45:27.334800+0000","last_undegraded":"2023-10-06T02:45:27.334800+0000","last_fullsized":"2023-10-06T02:45:27.334800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:52.328524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334307+0000","last_change":"2023-10-06T02:45:22.809096+0000","last_active":"2023-10-06T02:45:27.334307+0000","last_peered":"2023-10-06T02:45:27.334307+0000","last_clean":"2023-10-06T02:45:27.334307+0000","last_became_active":"2023-10-06T02:45:22.807367+0000","last_became_peered":"2023-10-06T02:45:22.807367+0000","last_unstale":"2023-10-06T02:45:27.334307+0000","last_undegraded":"2023-10-06T02:45:27.334307+0000","last_fullsized":"2023-10-06T02:45:27.334307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:34.291088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357950+0000","last_change":"2023-10-06T02:45:27.349954+0000","last_active":"2023-10-06T02:45:27.357950+0000","last_peered":"2023-10-06T02:45:27.357950+0000","last_clean":"2023-10-06T02:45:27.357950+0000","last_became_active":"2023-10-06T02:45:27.344601+0000","last_became_peered":"2023-10-06T02:45:27.344601+0000","last_unstale":"2023-10-06T02:45:27.357950+0000","last_undegraded":"2023-10-06T02:45:27.357950+0000","last_fullsized":"2023-10-06T02:45:27.357950+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:55.863350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340458+0000","last_change":"2023-10-06T02:45:23.325407+0000","last_active":"2023-10-06T02:45:27.340458+0000","last_peered":"2023-10-06T02:45:27.340458+0000","last_clean":"2023-10-06T02:45:27.340458+0000","last_became_active":"2023-10-06T02:45:23.325027+0000","last_became_peered":"2023-10-06T02:45:23.325027+0000","last_unstale":"2023-10-06T02:45:27.340458+0000","last_undegraded":"2023-10-06T02:45:27.340458+0000","last_fullsized":"2023-10-06T02:45:27.340458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334293+0000","last_change":"2023-10-06T02:45:22.807221+0000","last_active":"2023-10-06T02:45:27.334293+0000","last_peered":"2023-10-06T02:45:27.334293+0000","last_clean":"2023-10-06T02:45:27.334293+0000","last_became_active":"2023-10-06T02:45:22.806899+0000","last_became_peered":"2023-10-06T02:45:22.806899+0000","last_unstale":"2023-10-06T02:45:27.334293+0000","last_undegraded":"2023-10-06T02:45:27.334293+0000","last_fullsized":"2023-10-06T02:45:27.334293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:25.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.356009+0000","last_change":"2023-10-06T02:45:27.352601+0000","last_active":"2023-10-06T02:45:27.356009+0000","last_peered":"2023-10-06T02:45:27.356009+0000","last_clean":"2023-10-06T02:45:27.356009+0000","last_became_active":"2023-10-06T02:45:27.352177+0000","last_became_peered":"2023-10-06T02:45:27.352177+0000","last_unstale":"2023-10-06T02:45:27.356009+0000","last_undegraded":"2023-10-06T02:45:27.356009+0000","last_fullsized":"2023-10-06T02:45:27.356009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:57.927031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357070+0000","last_change":"2023-10-06T02:45:27.354742+0000","last_active":"2023-10-06T02:45:27.357070+0000","last_peered":"2023-10-06T02:45:27.357070+0000","last_clean":"2023-10-06T02:45:27.357070+0000","last_became_active":"2023-10-06T02:45:27.354154+0000","last_became_peered":"2023-10-06T02:45:27.354154+0000","last_unstale":"2023-10-06T02:45:27.357070+0000","last_undegraded":"2023-10-06T02:45:27.357070+0000","last_fullsized":"2023-10-06T02:45:27.357070+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:03.091381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.327140+0000","last_change":"2023-10-06T02:45:22.802084+0000","last_active":"2023-10-06T02:45:27.327140+0000","last_peered":"2023-10-06T02:45:27.327140+0000","last_clean":"2023-10-06T02:45:27.327140+0000","last_became_active":"2023-10-06T02:45:22.801685+0000","last_became_peered":"2023-10-06T02:45:22.801685+0000","last_unstale":"2023-10-06T02:45:27.327140+0000","last_undegraded":"2023-10-06T02:45:27.327140+0000","last_fullsized":"2023-10-06T02:45:27.327140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:31.821785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333425+0000","last_change":"2023-10-06T02:45:23.323734+0000","last_active":"2023-10-06T02:45:27.333425+0000","last_peered":"2023-10-06T02:45:27.333425+0000","last_clean":"2023-10-06T02:45:27.333425+0000","last_became_active":"2023-10-06T02:45:23.323390+0000","last_became_peered":"2023-10-06T02:45:23.323390+0000","last_unstale":"2023-10-06T02:45:27.333425+0000","last_undegraded":"2023-10-06T02:45:27.333425+0000","last_fullsized":"2023-10-06T02:45:27.333425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:09:04.032805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.350027+0000","last_change":"2023-10-06T02:45:27.346307+0000","last_active":"2023-10-06T02:45:27.350027+0000","last_peered":"2023-10-06T02:45:27.350027+0000","last_clean":"2023-10-06T02:45:27.350027+0000","last_became_active":"2023-10-06T02:45:27.345807+0000","last_became_peered":"2023-10-06T02:45:27.345807+0000","last_unstale":"2023-10-06T02:45:27.350027+0000","last_undegraded":"2023-10-06T02:45:27.350027+0000","last_fullsized":"2023-10-06T02:45:27.350027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.633950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332194+0000","last_change":"2023-10-06T02:45:22.809079+0000","last_active":"2023-10-06T02:45:27.332194+0000","last_peered":"2023-10-06T02:45:27.332194+0000","last_clean":"2023-10-06T02:45:27.332194+0000","last_became_active":"2023-10-06T02:45:22.807326+0000","last_became_peered":"2023-10-06T02:45:22.807326+0000","last_unstale":"2023-10-06T02:45:27.332194+0000","last_undegraded":"2023-10-06T02:45:27.332194+0000","last_fullsized":"2023-10-06T02:45:27.332194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:32.124040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334582+0000","last_change":"2023-10-06T02:45:23.313113+0000","last_active":"2023-10-06T02:45:27.334582+0000","last_peered":"2023-10-06T02:45:27.334582+0000","last_clean":"2023-10-06T02:45:27.334582+0000","last_became_active":"2023-10-06T02:45:23.312797+0000","last_became_peered":"2023-10-06T02:45:23.312797+0000","last_unstale":"2023-10-06T02:45:27.334582+0000","last_undegraded":"2023-10-06T02:45:27.334582+0000","last_fullsized":"2023-10-06T02:45:27.334582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:22.191471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334672+0000","last_change":"2023-10-06T02:45:22.804236+0000","last_active":"2023-10-06T02:45:27.334672+0000","last_peered":"2023-10-06T02:45:27.334672+0000","last_clean":"2023-10-06T02:45:27.334672+0000","last_became_active":"2023-10-06T02:45:22.802766+0000","last_became_peered":"2023-10-06T02:45:22.802766+0000","last_unstale":"2023-10-06T02:45:27.334672+0000","last_undegraded":"2023-10-06T02:45:27.334672+0000","last_fullsized":"2023-10-06T02:45:27.334672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.084999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341457+0000","last_change":"2023-10-06T02:45:23.324554+0000","last_active":"2023-10-06T02:45:27.341457+0000","last_peered":"2023-10-06T02:45:27.341457+0000","last_clean":"2023-10-06T02:45:27.341457+0000","last_became_active":"2023-10-06T02:45:23.324267+0000","last_became_peered":"2023-10-06T02:45:23.324267+0000","last_unstale":"2023-10-06T02:45:27.341457+0000","last_undegraded":"2023-10-06T02:45:27.341457+0000","last_fullsized":"2023-10-06T02:45:27.341457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.354813+0000","last_change":"2023-10-06T02:45:27.346623+0000","last_active":"2023-10-06T02:45:27.354813+0000","last_peered":"2023-10-06T02:45:27.354813+0000","last_clean":"2023-10-06T02:45:27.354813+0000","last_became_active":"2023-10-06T02:45:27.346022+0000","last_became_peered":"2023-10-06T02:45:27.346022+0000","last_unstale":"2023-10-06T02:45:27.354813+0000","last_undegraded":"2023-10-06T02:45:27.354813+0000","last_fullsized":"2023-10-06T02:45:27.354813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:10:40.724648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332912+0000","last_change":"2023-10-06T02:45:22.805769+0000","last_active":"2023-10-06T02:45:27.332912+0000","last_peered":"2023-10-06T02:45:27.332912+0000","last_clean":"2023-10-06T02:45:27.332912+0000","last_became_active":"2023-10-06T02:45:22.805441+0000","last_became_peered":"2023-10-06T02:45:22.805441+0000","last_unstale":"2023-10-06T02:45:27.332912+0000","last_undegraded":"2023-10-06T02:45:27.332912+0000","last_fullsized":"2023-10-06T02:45:27.332912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:21.761279+0000","last_clean_scrub_stamp":"2023-10-06T02:45:21.761279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:41.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.338579+0000","last_change":"2023-10-06T02:45:23.312634+0000","last_active":"2023-10-06T02:45:27.338579+0000","last_peered":"2023-10-06T02:45:27.338579+0000","last_clean":"2023-10-06T02:45:27.338579+0000","last_became_active":"2023-10-06T02:45:23.312292+0000","last_became_peered":"2023-10-06T02:45:23.312292+0000","last_unstale":"2023-10-06T02:45:27.338579+0000","last_undegraded":"2023-10-06T02:45:27.338579+0000","last_fullsized":"2023-10-06T02:45:27.338579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:05.795770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.357104+0000","last_change":"2023-10-06T02:45:27.354353+0000","last_active":"2023-10-06T02:45:27.357104+0000","last_peered":"2023-10-06T02:45:27.357104+0000","last_clean":"2023-10-06T02:45:27.357104+0000","last_became_active":"2023-10-06T02:45:27.353428+0000","last_became_peered":"2023-10-06T02:45:27.353428+0000","last_unstale":"2023-10-06T02:45:27.357104+0000","last_undegraded":"2023-10-06T02:45:27.357104+0000","last_fullsized":"2023-10-06T02:45:27.357104+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:26.300626+0000","last_clean_scrub_stamp":"2023-10-06T02:45:26.300626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.149006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325192+0000","last_change":"2023-10-06T02:45:23.309222+0000","last_active":"2023-10-06T02:45:27.325192+0000","last_peered":"2023-10-06T02:45:27.325192+0000","last_clean":"2023-10-06T02:45:27.325192+0000","last_became_active":"2023-10-06T02:45:23.308920+0000","last_became_peered":"2023-10-06T02:45:23.308920+0000","last_unstale":"2023-10-06T02:45:27.325192+0000","last_undegraded":"2023-10-06T02:45:27.325192+0000","last_fullsized":"2023-10-06T02:45:27.325192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:41.399143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329866+0000","last_change":"2023-10-06T02:45:23.312534+0000","last_active":"2023-10-06T02:45:27.329866+0000","last_peered":"2023-10-06T02:45:27.329866+0000","last_clean":"2023-10-06T02:45:27.329866+0000","last_became_active":"2023-10-06T02:45:23.312215+0000","last_became_peered":"2023-10-06T02:45:23.312215+0000","last_unstale":"2023-10-06T02:45:27.329866+0000","last_undegraded":"2023-10-06T02:45:27.329866+0000","last_fullsized":"2023-10-06T02:45:27.329866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:39.416511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336549+0000","last_change":"2023-10-06T02:45:23.305501+0000","last_active":"2023-10-06T02:45:27.336549+0000","last_peered":"2023-10-06T02:45:27.336549+0000","last_clean":"2023-10-06T02:45:27.336549+0000","last_became_active":"2023-10-06T02:45:23.305062+0000","last_became_peered":"2023-10-06T02:45:23.305062+0000","last_unstale":"2023-10-06T02:45:27.336549+0000","last_undegraded":"2023-10-06T02:45:27.336549+0000","last_fullsized":"2023-10-06T02:45:27.336549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.333770+0000","last_change":"2023-10-06T02:45:23.324370+0000","last_active":"2023-10-06T02:45:27.333770+0000","last_peered":"2023-10-06T02:45:27.333770+0000","last_clean":"2023-10-06T02:45:27.333770+0000","last_became_active":"2023-10-06T02:45:23.324082+0000","last_became_peered":"2023-10-06T02:45:23.324082+0000","last_unstale":"2023-10-06T02:45:27.333770+0000","last_undegraded":"2023-10-06T02:45:27.333770+0000","last_fullsized":"2023-10-06T02:45:27.333770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:35.146269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.337261+0000","last_change":"2023-10-06T02:45:23.322276+0000","last_active":"2023-10-06T02:45:27.337261+0000","last_peered":"2023-10-06T02:45:27.337261+0000","last_clean":"2023-10-06T02:45:27.337261+0000","last_became_active":"2023-10-06T02:45:23.321381+0000","last_became_peered":"2023-10-06T02:45:23.321381+0000","last_unstale":"2023-10-06T02:45:27.337261+0000","last_undegraded":"2023-10-06T02:45:27.337261+0000","last_fullsized":"2023-10-06T02:45:27.337261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341408+0000","last_change":"2023-10-06T02:45:23.324552+0000","last_active":"2023-10-06T02:45:27.341408+0000","last_peered":"2023-10-06T02:45:27.341408+0000","last_clean":"2023-10-06T02:45:27.341408+0000","last_became_active":"2023-10-06T02:45:23.324145+0000","last_became_peered":"2023-10-06T02:45:23.324145+0000","last_unstale":"2023-10-06T02:45:27.341408+0000","last_undegraded":"2023-10-06T02:45:27.341408+0000","last_fullsized":"2023-10-06T02:45:27.341408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340251+0000","last_change":"2023-10-06T02:45:23.316105+0000","last_active":"2023-10-06T02:45:27.340251+0000","last_peered":"2023-10-06T02:45:27.340251+0000","last_clean":"2023-10-06T02:45:27.340251+0000","last_became_active":"2023-10-06T02:45:23.315133+0000","last_became_peered":"2023-10-06T02:45:23.315133+0000","last_unstale":"2023-10-06T02:45:27.340251+0000","last_undegraded":"2023-10-06T02:45:27.340251+0000","last_fullsized":"2023-10-06T02:45:27.340251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:02.002661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.339997+0000","last_change":"2023-10-06T02:45:23.315937+0000","last_active":"2023-10-06T02:45:27.339997+0000","last_peered":"2023-10-06T02:45:27.339997+0000","last_clean":"2023-10-06T02:45:27.339997+0000","last_became_active":"2023-10-06T02:45:23.315541+0000","last_became_peered":"2023-10-06T02:45:23.315541+0000","last_unstale":"2023-10-06T02:45:27.339997+0000","last_undegraded":"2023-10-06T02:45:27.339997+0000","last_fullsized":"2023-10-06T02:45:27.339997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331530+0000","last_change":"2023-10-06T02:45:23.326576+0000","last_active":"2023-10-06T02:45:27.331530+0000","last_peered":"2023-10-06T02:45:27.331530+0000","last_clean":"2023-10-06T02:45:27.331530+0000","last_became_active":"2023-10-06T02:45:23.326298+0000","last_became_peered":"2023-10-06T02:45:23.326298+0000","last_unstale":"2023-10-06T02:45:27.331530+0000","last_undegraded":"2023-10-06T02:45:27.331530+0000","last_fullsized":"2023-10-06T02:45:27.331530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.329766+0000","last_change":"2023-10-06T02:45:23.315079+0000","last_active":"2023-10-06T02:45:27.329766+0000","last_peered":"2023-10-06T02:45:27.329766+0000","last_clean":"2023-10-06T02:45:27.329766+0000","last_became_active":"2023-10-06T02:45:23.314780+0000","last_became_peered":"2023-10-06T02:45:23.314780+0000","last_unstale":"2023-10-06T02:45:27.329766+0000","last_undegraded":"2023-10-06T02:45:27.329766+0000","last_fullsized":"2023-10-06T02:45:27.329766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:22.869594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334578+0000","last_change":"2023-10-06T02:45:23.314883+0000","last_active":"2023-10-06T02:45:27.334578+0000","last_peered":"2023-10-06T02:45:27.334578+0000","last_clean":"2023-10-06T02:45:27.334578+0000","last_became_active":"2023-10-06T02:45:23.314594+0000","last_became_peered":"2023-10-06T02:45:23.314594+0000","last_unstale":"2023-10-06T02:45:27.334578+0000","last_undegraded":"2023-10-06T02:45:27.334578+0000","last_fullsized":"2023-10-06T02:45:27.334578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332695+0000","last_change":"2023-10-06T02:45:23.324062+0000","last_active":"2023-10-06T02:45:27.332695+0000","last_peered":"2023-10-06T02:45:27.332695+0000","last_clean":"2023-10-06T02:45:27.332695+0000","last_became_active":"2023-10-06T02:45:23.323735+0000","last_became_peered":"2023-10-06T02:45:23.323735+0000","last_unstale":"2023-10-06T02:45:27.332695+0000","last_undegraded":"2023-10-06T02:45:27.332695+0000","last_fullsized":"2023-10-06T02:45:27.332695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336547+0000","last_change":"2023-10-06T02:45:23.306965+0000","last_active":"2023-10-06T02:45:27.336547+0000","last_peered":"2023-10-06T02:45:27.336547+0000","last_clean":"2023-10-06T02:45:27.336547+0000","last_became_active":"2023-10-06T02:45:23.306269+0000","last_became_peered":"2023-10-06T02:45:23.306269+0000","last_unstale":"2023-10-06T02:45:27.336547+0000","last_undegraded":"2023-10-06T02:45:27.336547+0000","last_fullsized":"2023-10-06T02:45:27.336547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320250+0000","last_change":"2023-10-06T02:45:23.305622+0000","last_active":"2023-10-06T02:45:27.320250+0000","last_peered":"2023-10-06T02:45:27.320250+0000","last_clean":"2023-10-06T02:45:27.320250+0000","last_became_active":"2023-10-06T02:45:23.305323+0000","last_became_peered":"2023-10-06T02:45:23.305323+0000","last_unstale":"2023-10-06T02:45:27.320250+0000","last_undegraded":"2023-10-06T02:45:27.320250+0000","last_fullsized":"2023-10-06T02:45:27.320250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:24.265950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334435+0000","last_change":"2023-10-06T02:45:23.311009+0000","last_active":"2023-10-06T02:45:27.334435+0000","last_peered":"2023-10-06T02:45:27.334435+0000","last_clean":"2023-10-06T02:45:27.334435+0000","last_became_active":"2023-10-06T02:45:23.310671+0000","last_became_peered":"2023-10-06T02:45:23.310671+0000","last_unstale":"2023-10-06T02:45:27.334435+0000","last_undegraded":"2023-10-06T02:45:27.334435+0000","last_fullsized":"2023-10-06T02:45:27.334435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:06.358484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334894+0000","last_change":"2023-10-06T02:45:23.312474+0000","last_active":"2023-10-06T02:45:27.334894+0000","last_peered":"2023-10-06T02:45:27.334894+0000","last_clean":"2023-10-06T02:45:27.334894+0000","last_became_active":"2023-10-06T02:45:23.312147+0000","last_became_peered":"2023-10-06T02:45:23.312147+0000","last_unstale":"2023-10-06T02:45:27.334894+0000","last_undegraded":"2023-10-06T02:45:27.334894+0000","last_fullsized":"2023-10-06T02:45:27.334894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:06.847673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.330934+0000","last_change":"2023-10-06T02:45:23.314526+0000","last_active":"2023-10-06T02:45:27.330934+0000","last_peered":"2023-10-06T02:45:27.330934+0000","last_clean":"2023-10-06T02:45:27.330934+0000","last_became_active":"2023-10-06T02:45:23.314229+0000","last_became_peered":"2023-10-06T02:45:23.314229+0000","last_unstale":"2023-10-06T02:45:27.330934+0000","last_undegraded":"2023-10-06T02:45:27.330934+0000","last_fullsized":"2023-10-06T02:45:27.330934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:08.831291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.332089+0000","last_change":"2023-10-06T02:45:23.326220+0000","last_active":"2023-10-06T02:45:27.332089+0000","last_peered":"2023-10-06T02:45:27.332089+0000","last_clean":"2023-10-06T02:45:27.332089+0000","last_became_active":"2023-10-06T02:45:23.325853+0000","last_became_peered":"2023-10-06T02:45:23.325853+0000","last_unstale":"2023-10-06T02:45:27.332089+0000","last_undegraded":"2023-10-06T02:45:27.332089+0000","last_fullsized":"2023-10-06T02:45:27.332089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:05.681146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340313+0000","last_change":"2023-10-06T02:45:23.319007+0000","last_active":"2023-10-06T02:45:27.340313+0000","last_peered":"2023-10-06T02:45:27.340313+0000","last_clean":"2023-10-06T02:45:27.340313+0000","last_became_active":"2023-10-06T02:45:23.318521+0000","last_became_peered":"2023-10-06T02:45:23.318521+0000","last_unstale":"2023-10-06T02:45:27.340313+0000","last_undegraded":"2023-10-06T02:45:27.340313+0000","last_fullsized":"2023-10-06T02:45:27.340313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:26.970108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319562+0000","last_change":"2023-10-06T02:45:23.308224+0000","last_active":"2023-10-06T02:45:27.319562+0000","last_peered":"2023-10-06T02:45:27.319562+0000","last_clean":"2023-10-06T02:45:27.319562+0000","last_became_active":"2023-10-06T02:45:23.307908+0000","last_became_peered":"2023-10-06T02:45:23.307908+0000","last_unstale":"2023-10-06T02:45:27.319562+0000","last_undegraded":"2023-10-06T02:45:27.319562+0000","last_fullsized":"2023-10-06T02:45:27.319562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:26.518844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.319631+0000","last_change":"2023-10-06T02:45:23.312138+0000","last_active":"2023-10-06T02:45:27.319631+0000","last_peered":"2023-10-06T02:45:27.319631+0000","last_clean":"2023-10-06T02:45:27.319631+0000","last_became_active":"2023-10-06T02:45:23.311612+0000","last_became_peered":"2023-10-06T02:45:23.311612+0000","last_unstale":"2023-10-06T02:45:27.319631+0000","last_undegraded":"2023-10-06T02:45:27.319631+0000","last_fullsized":"2023-10-06T02:45:27.319631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:29.807177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336407+0000","last_change":"2023-10-06T02:45:23.314057+0000","last_active":"2023-10-06T02:45:27.336407+0000","last_peered":"2023-10-06T02:45:27.336407+0000","last_clean":"2023-10-06T02:45:27.336407+0000","last_became_active":"2023-10-06T02:45:23.313762+0000","last_became_peered":"2023-10-06T02:45:23.313762+0000","last_unstale":"2023-10-06T02:45:27.336407+0000","last_undegraded":"2023-10-06T02:45:27.336407+0000","last_fullsized":"2023-10-06T02:45:27.336407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:54.392205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341622+0000","last_change":"2023-10-06T02:45:23.318996+0000","last_active":"2023-10-06T02:45:27.341622+0000","last_peered":"2023-10-06T02:45:27.341622+0000","last_clean":"2023-10-06T02:45:27.341622+0000","last_became_active":"2023-10-06T02:45:23.318710+0000","last_became_peered":"2023-10-06T02:45:23.318710+0000","last_unstale":"2023-10-06T02:45:27.341622+0000","last_undegraded":"2023-10-06T02:45:27.341622+0000","last_fullsized":"2023-10-06T02:45:27.341622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:04.789619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.334795+0000","last_change":"2023-10-06T02:45:23.312737+0000","last_active":"2023-10-06T02:45:27.334795+0000","last_peered":"2023-10-06T02:45:27.334795+0000","last_clean":"2023-10-06T02:45:27.334795+0000","last_became_active":"2023-10-06T02:45:23.312446+0000","last_became_peered":"2023-10-06T02:45:23.312446+0000","last_unstale":"2023-10-06T02:45:27.334795+0000","last_undegraded":"2023-10-06T02:45:27.334795+0000","last_fullsized":"2023-10-06T02:45:27.334795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:01.582052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.340404+0000","last_change":"2023-10-06T02:45:23.318887+0000","last_active":"2023-10-06T02:45:27.340404+0000","last_peered":"2023-10-06T02:45:27.340404+0000","last_clean":"2023-10-06T02:45:27.340404+0000","last_became_active":"2023-10-06T02:45:23.318573+0000","last_became_peered":"2023-10-06T02:45:23.318573+0000","last_unstale":"2023-10-06T02:45:27.340404+0000","last_undegraded":"2023-10-06T02:45:27.340404+0000","last_fullsized":"2023-10-06T02:45:27.340404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:41.025954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.331986+0000","last_change":"2023-10-06T02:45:23.326294+0000","last_active":"2023-10-06T02:45:27.331986+0000","last_peered":"2023-10-06T02:45:27.331986+0000","last_clean":"2023-10-06T02:45:27.331986+0000","last_became_active":"2023-10-06T02:45:23.325935+0000","last_became_peered":"2023-10-06T02:45:23.325935+0000","last_unstale":"2023-10-06T02:45:27.331986+0000","last_undegraded":"2023-10-06T02:45:27.331986+0000","last_fullsized":"2023-10-06T02:45:27.331986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:20.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336417+0000","last_change":"2023-10-06T02:45:23.316621+0000","last_active":"2023-10-06T02:45:27.336417+0000","last_peered":"2023-10-06T02:45:27.336417+0000","last_clean":"2023-10-06T02:45:27.336417+0000","last_became_active":"2023-10-06T02:45:23.316283+0000","last_became_peered":"2023-10-06T02:45:23.316283+0000","last_unstale":"2023-10-06T02:45:27.336417+0000","last_undegraded":"2023-10-06T02:45:27.336417+0000","last_fullsized":"2023-10-06T02:45:27.336417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:21.187543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.335121+0000","last_change":"2023-10-06T02:45:23.318488+0000","last_active":"2023-10-06T02:45:27.335121+0000","last_peered":"2023-10-06T02:45:27.335121+0000","last_clean":"2023-10-06T02:45:27.335121+0000","last_became_active":"2023-10-06T02:45:23.318144+0000","last_became_peered":"2023-10-06T02:45:23.318144+0000","last_unstale":"2023-10-06T02:45:27.335121+0000","last_undegraded":"2023-10-06T02:45:27.335121+0000","last_fullsized":"2023-10-06T02:45:27.335121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:03.340355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.325094+0000","last_change":"2023-10-06T02:45:23.316019+0000","last_active":"2023-10-06T02:45:27.325094+0000","last_peered":"2023-10-06T02:45:27.325094+0000","last_clean":"2023-10-06T02:45:27.325094+0000","last_became_active":"2023-10-06T02:45:23.315706+0000","last_became_peered":"2023-10-06T02:45:23.315706+0000","last_unstale":"2023-10-06T02:45:27.325094+0000","last_undegraded":"2023-10-06T02:45:27.325094+0000","last_fullsized":"2023-10-06T02:45:27.325094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:57.498422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.336118+0000","last_change":"2023-10-06T02:45:23.313938+0000","last_active":"2023-10-06T02:45:27.336118+0000","last_peered":"2023-10-06T02:45:27.336118+0000","last_clean":"2023-10-06T02:45:27.336118+0000","last_became_active":"2023-10-06T02:45:23.313340+0000","last_became_peered":"2023-10-06T02:45:23.313340+0000","last_unstale":"2023-10-06T02:45:27.336118+0000","last_undegraded":"2023-10-06T02:45:27.336118+0000","last_fullsized":"2023-10-06T02:45:27.336118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:53.828654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.341529+0000","last_change":"2023-10-06T02:45:23.321544+0000","last_active":"2023-10-06T02:45:27.341529+0000","last_peered":"2023-10-06T02:45:27.341529+0000","last_clean":"2023-10-06T02:45:27.341529+0000","last_became_active":"2023-10-06T02:45:23.321216+0000","last_became_peered":"2023-10-06T02:45:23.321216+0000","last_unstale":"2023-10-06T02:45:27.341529+0000","last_undegraded":"2023-10-06T02:45:27.341529+0000","last_fullsized":"2023-10-06T02:45:27.341529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:15.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T02:45:27.320111+0000","last_change":"2023-10-06T02:45:23.309006+0000","last_active":"2023-10-06T02:45:27.320111+0000","last_peered":"2023-10-06T02:45:27.320111+0000","last_clean":"2023-10-06T02:45:27.320111+0000","last_became_active":"2023-10-06T02:45:23.308617+0000","last_became_peered":"2023-10-06T02:45:23.308617+0000","last_unstale":"2023-10-06T02:45:27.320111+0000","last_undegraded":"2023-10-06T02:45:27.320111+0000","last_fullsized":"2023-10-06T02:45:27.320111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:22.765800+0000","last_clean_scrub_stamp":"2023-10-06T02:45:22.765800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:35.295610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4484,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]}]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":131211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":315392,"data_stored":127531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":127441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":718363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:45:46.838 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T02:45:46.838 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T02:45:46.838 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T02:45:46.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T02:45:46.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.969+0000 7fa07cbe5700 1 Processor -- start 2023-10-06T02:45:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.969+0000 7fa07cbe5700 1 -- start start 2023-10-06T02:45:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0780cc820 0x7fa0780ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 0x7fa078153100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07815cc80 con 0x7fa07815d110 2023-10-06T02:45:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0780cb2e0 con 0x7fa0780cc820 2023-10-06T02:45:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa078153640 con 0x7fa078152c80 2023-10-06T02:45:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 0x7fa078153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa0777fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0780cc820 0x7fa0780ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 0x7fa078153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54858/0 (socket says 172.21.15.33:54858) 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa0777fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa0780cc820 0x7fa0780ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:43968/0 (socket says 172.21.15.33:43968) 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 -- 172.21.15.33:0/800479069 learned_addr learned my addr 172.21.15.33:0/800479069 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa0767fc700 1 -- 172.21.15.33:0/800479069 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466865009 0 0) 0x7fa07815cc80 con 0x7fa07815d110 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 -- 172.21.15.33:0/800479069 >> v2:172.21.15.167:3300/0 conn(0x7fa0780cc820 msgr2=0x7fa0780ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 --2- 172.21.15.33:0/800479069 >> v2:172.21.15.167:3300/0 conn(0x7fa0780cc820 0x7fa0780ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 -- 172.21.15.33:0/800479069 >> v1:172.21.15.167:6789/0 conn(0x7fa07815d110 legacy=0x7fa0780cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 -- 172.21.15.33:0/800479069 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0780cf4d0 con 0x7fa078152c80 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa077fff700 1 --2- 172.21.15.33:0/800479069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 0x7fa078153100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0700099b0 tx=0x7fa070009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c40da53c2782d0ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa0767fc700 1 -- 172.21.15.33:0/800479069 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070009ea0 con 0x7fa078152c80 2023-10-06T02:45:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa0767fc700 1 -- 172.21.15.33:0/800479069 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07000b860 con 0x7fa078152c80 2023-10-06T02:45:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa0767fc700 1 -- 172.21.15.33:0/800479069 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07000bc00 con 0x7fa078152c80 2023-10-06T02:45:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- 172.21.15.33:0/800479069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 msgr2=0x7fa078153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/800479069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 0x7fa078153100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0700099b0 tx=0x7fa070009250 comp rx=0 tx=0).stop 2023-10-06T02:45:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- 172.21.15.33:0/800479069 shutdown_connections 2023-10-06T02:45:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/800479069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa078152c80 0x7fa078153100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/800479069 >> v2:172.21.15.167:3300/0 conn(0x7fa0780cc820 0x7fa0780ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- 172.21.15.33:0/800479069 >> 172.21.15.33:0/800479069 conn(0x7fa078066fb0 msgr2=0x7fa0780673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:46.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- 172.21.15.33:0/800479069 shutdown_connections 2023-10-06T02:45:46.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.973+0000 7fa07cbe5700 1 -- 172.21.15.33:0/800479069 wait complete. 2023-10-06T02:45:46.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 Processor -- start 2023-10-06T02:45:46.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 -- start start 2023-10-06T02:45:46.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0780cc820 0x7fa0780b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 0x7fa078144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa07815d110 0x7fa07814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07815a4c0 con 0x7fa078152c80 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0780b14b0 con 0x7fa07815d110 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0780b0970 con 0x7fa0780cc820 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 0x7fa078144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 0x7fa078144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54950/0 (socket says 172.21.15.33:54950) 2023-10-06T02:45:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 -- 172.21.15.33:0/710633671 learned_addr learned my addr 172.21.15.33:0/710633671 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:46.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa077fff700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa07815d110 0x7fa07814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa0777fe700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0780cc820 0x7fa0780b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 -- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0780cc820 msgr2=0x7fa0780b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0780cc820 0x7fa0780b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 -- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa07815d110 msgr2=0x7fa07814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:46.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa07815d110 0x7fa07814d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:46.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 -- 172.21.15.33:0/710633671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700094f0 con 0x7fa078152c80 2023-10-06T02:45:46.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa076ffd700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 0x7fa078144900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa06000c910 tx=0x7fa060001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060012630 con 0x7fa078152c80 2023-10-06T02:45:46.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060012790 con 0x7fa078152c80 2023-10-06T02:45:46.987 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0780b0c10 con 0x7fa078152c80 2023-10-06T02:45:46.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060012b50 con 0x7fa078152c80 2023-10-06T02:45:46.988 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.977+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0780b11d0 con 0x7fa078152c80 2023-10-06T02:45:46.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.981+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07815a670 con 0x7fa078152c80 2023-10-06T02:45:46.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.981+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa060003030 con 0x7fa078152c80 2023-10-06T02:45:46.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.981+0000 7fa06f7fe700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa058060550 0x7fa058062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:46.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.981+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa06007cc60 con 0x7fa078152c80 2023-10-06T02:45:46.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.981+0000 7fa0777fe700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa058060550 0x7fa058062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:46.989 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.981+0000 7fa0777fe700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa058060550 0x7fa058062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa068009d40 tx=0x7fa068008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:46.989+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa07815a670 con 0x7fa078152c80 2023-10-06T02:45:47.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.173+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa078071040 con 0x7fa078152c80 2023-10-06T02:45:47.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.173+0000 7fa06f7fe700 1 -- 172.21.15.33:0/710633671 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa06004ce00 con 0x7fa078152c80 2023-10-06T02:45:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:47.180 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T02:45:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa058060550 msgr2=0x7fa058062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa058060550 0x7fa058062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa068009d40 tx=0x7fa068008040 comp rx=0 tx=0).stop 2023-10-06T02:45:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 msgr2=0x7fa078144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 0x7fa078144900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa06000c910 tx=0x7fa060001700 comp rx=0 tx=0).stop 2023-10-06T02:45:47.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 shutdown_connections 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa07815d110 0x7fa07814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078152c80 0x7fa078144900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa058060550 0x7fa058062a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 --2- 172.21.15.33:0/710633671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0780cc820 0x7fa0780b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 >> 172.21.15.33:0/710633671 conn(0x7fa078066fb0 msgr2=0x7fa0780ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 shutdown_connections 2023-10-06T02:45:47.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.177+0000 7fa07cbe5700 1 -- 172.21.15.33:0/710633671 wait complete. 2023-10-06T02:45:47.193 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T02:45:47.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.289+0000 7f34e985e700 1 Processor -- start 2023-10-06T02:45:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.289+0000 7f34e985e700 1 -- start start 2023-10-06T02:45:47.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f34dc036f20 0x7f34dc0372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0342e0 0x7f34dc007380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f34dc04faa0 0x7f34dc04ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc006090 con 0x7f34dc04faa0 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc006ef0 con 0x7f34dc036f20 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0062e0 con 0x7f34dc0342e0 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f34dc04faa0 0x7f34dc04ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f34dc04faa0 0x7f34dc04ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54964/0 (socket says 172.21.15.33:54964) 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 -- 172.21.15.33:0/719700945 learned_addr learned my addr 172.21.15.33:0/719700945 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e3fff700 1 --2- 172.21.15.33:0/719700945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0342e0 0x7f34dc007380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e885c700 1 --2- 172.21.15.33:0/719700945 >> v2:172.21.15.167:3300/0 conn(0x7f34dc036f20 0x7f34dc0372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 -- 172.21.15.33:0/719700945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0342e0 msgr2=0x7f34dc007380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 --2- 172.21.15.33:0/719700945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0342e0 0x7f34dc007380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 -- 172.21.15.33:0/719700945 >> v2:172.21.15.167:3300/0 conn(0x7f34dc036f20 msgr2=0x7f34dc0372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 --2- 172.21.15.33:0/719700945 >> v2:172.21.15.167:3300/0 conn(0x7f34dc036f20 0x7f34dc0372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 -- 172.21.15.33:0/719700945 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0065c0 con 0x7f34dc04faa0 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e885c700 1 --2- 172.21.15.33:0/719700945 >> v2:172.21.15.167:3300/0 conn(0x7f34dc036f20 0x7f34dc0372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e905d700 1 --2- 172.21.15.33:0/719700945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc04faa0 0x7f34dc04ff00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34dc055cc0 tx=0x7f34e4070a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76d16f0d849c499 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e37fe700 1 -- 172.21.15.33:0/719700945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4069070 con 0x7f34dc04faa0 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e37fe700 1 -- 172.21.15.33:0/719700945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e406a040 con 0x7f34dc04faa0 2023-10-06T02:45:47.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e37fe700 1 -- 172.21.15.33:0/719700945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e406f9a0 con 0x7f34dc04faa0 2023-10-06T02:45:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- 172.21.15.33:0/719700945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc04faa0 msgr2=0x7f34dc04ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- 172.21.15.33:0/719700945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc04faa0 0x7f34dc04ff00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34dc055cc0 tx=0x7f34e4070a50 comp rx=0 tx=0).stop 2023-10-06T02:45:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- 172.21.15.33:0/719700945 shutdown_connections 2023-10-06T02:45:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- 172.21.15.33:0/719700945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc04faa0 0x7f34dc04ff00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- 172.21.15.33:0/719700945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc0342e0 0x7f34dc007380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 --2- 172.21.15.33:0/719700945 >> v2:172.21.15.167:3300/0 conn(0x7f34dc036f20 0x7f34dc0372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- 172.21.15.33:0/719700945 >> 172.21.15.33:0/719700945 conn(0x7f34dc054e00 msgr2=0x7f34dc055200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- 172.21.15.33:0/719700945 shutdown_connections 2023-10-06T02:45:47.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.293+0000 7f34e985e700 1 -- 172.21.15.33:0/719700945 wait complete. 2023-10-06T02:45:47.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 Processor -- start 2023-10-06T02:45:47.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 -- start start 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0342e0 0x7f34dc0d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 0x7f34dc0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc04faa0 0x7f34dc0d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0d2a30 con 0x7f34dc036f20 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc04d7d0 con 0x7f34dc0342e0 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.297+0000 7f34e985e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0d1e20 con 0x7f34dc04faa0 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 0x7f34dc0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 0x7f34dc0d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54978/0 (socket says 172.21.15.33:54978) 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 -- 172.21.15.33:0/1878493425 learned_addr learned my addr 172.21.15.33:0/1878493425 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e885c700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0342e0 0x7f34dc0d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e905d700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc04faa0 0x7f34dc0d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 -- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc04faa0 msgr2=0x7f34dc0d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc04faa0 0x7f34dc0d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 -- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0342e0 msgr2=0x7f34dc0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0342e0 0x7f34dc0d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 -- 172.21.15.33:0/1878493425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4063bb0 con 0x7f34dc036f20 2023-10-06T02:45:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e905d700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc04faa0 0x7f34dc0d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e3fff700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 0x7f34dc0d12b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34d000d890 tx=0x7f34d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d00125f0 con 0x7f34dc036f20 2023-10-06T02:45:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d0012770 con 0x7f34dc036f20 2023-10-06T02:45:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc0d2100 con 0x7f34dc036f20 2023-10-06T02:45:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0012b30 con 0x7f34dc036f20 2023-10-06T02:45:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc0d26c0 con 0x7f34dc036f20 2023-10-06T02:45:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc0d2be0 con 0x7f34dc036f20 2023-10-06T02:45:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.301+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f34d001e460 con 0x7f34dc036f20 2023-10-06T02:45:47.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.305+0000 7f34e0ff9700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f34c8060550 0x7f34c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.305+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f34d007d270 con 0x7f34dc036f20 2023-10-06T02:45:47.309 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.305+0000 7f34e885c700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f34c8060550 0x7f34c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.305+0000 7f34e885c700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f34c8060550 0x7f34c8062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f34d800adc0 tx=0x7f34d8006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.313+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f34d004d410 con 0x7f34dc036f20 2023-10-06T02:45:47.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.473+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34dc005500 con 0x7f34dc036f20 2023-10-06T02:45:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e0ff9700 1 -- 172.21.15.33:0/1878493425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f34d0050a30 con 0x7f34dc036f20 2023-10-06T02:45:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:47.481 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f34c8060550 msgr2=0x7f34c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f34c8060550 0x7f34c8062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f34d800adc0 tx=0x7f34d8006f90 comp rx=0 tx=0).stop 2023-10-06T02:45:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 msgr2=0x7f34dc0d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 0x7f34dc0d12b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34d000d890 tx=0x7f34d0008040 comp rx=0 tx=0).stop 2023-10-06T02:45:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 shutdown_connections 2023-10-06T02:45:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f34c8060550 0x7f34c8062a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34dc04faa0 0x7f34dc0d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34dc036f20 0x7f34dc0d12b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 --2- 172.21.15.33:0/1878493425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34dc0342e0 0x7f34dc0d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 >> 172.21.15.33:0/1878493425 conn(0x7f34dc054e00 msgr2=0x7f34dc052ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 shutdown_connections 2023-10-06T02:45:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.477+0000 7f34e985e700 1 -- 172.21.15.33:0/1878493425 wait complete. 2023-10-06T02:45:47.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:45:47.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:45:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 Processor -- start 2023-10-06T02:45:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 -- start start 2023-10-06T02:45:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2550154e20 0x7f25500caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550066fb0 0x7f25500cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f25501551f0 0x7f255006a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500c7590 con 0x7f2550154e20 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255006b230 con 0x7f25501551f0 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2556fcf700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255006b510 con 0x7f2550066fb0 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2550154e20 0x7f25500caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2550154e20 0x7f25500caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54982/0 (socket says 172.21.15.33:54982) 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 -- 172.21.15.33:0/1264169814 learned_addr learned my addr 172.21.15.33:0/1264169814 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f25557cc700 1 --2- 172.21.15.33:0/1264169814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550066fb0 0x7f25500cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f25567ce700 1 --2- 172.21.15.33:0/1264169814 >> v2:172.21.15.167:3300/0 conn(0x7f25501551f0 0x7f255006a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 -- 172.21.15.33:0/1264169814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550066fb0 msgr2=0x7f25500cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 --2- 172.21.15.33:0/1264169814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550066fb0 0x7f25500cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 -- 172.21.15.33:0/1264169814 >> v2:172.21.15.167:3300/0 conn(0x7f25501551f0 msgr2=0x7f255006a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 --2- 172.21.15.33:0/1264169814 >> v2:172.21.15.167:3300/0 conn(0x7f25501551f0 0x7f255006a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.613+0000 7f2555fcd700 1 -- 172.21.15.33:0/1264169814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255006b7f0 con 0x7f2550154e20 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2555fcd700 1 --2- 172.21.15.33:0/1264169814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550154e20 0x7f25500caa70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f25500c3050 tx=0x7f254c00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963ed4ac95002505 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2554fcb700 1 -- 172.21.15.33:0/1264169814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00f070 con 0x7f2550154e20 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2554fcb700 1 -- 172.21.15.33:0/1264169814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c00cdc0 con 0x7f2550154e20 2023-10-06T02:45:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2554fcb700 1 -- 172.21.15.33:0/1264169814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00a320 con 0x7f2550154e20 2023-10-06T02:45:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- 172.21.15.33:0/1264169814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550154e20 msgr2=0x7f25500caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- 172.21.15.33:0/1264169814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550154e20 0x7f25500caa70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f25500c3050 tx=0x7f254c00a950 comp rx=0 tx=0).stop 2023-10-06T02:45:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- 172.21.15.33:0/1264169814 shutdown_connections 2023-10-06T02:45:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- 172.21.15.33:0/1264169814 >> v2:172.21.15.167:3300/0 conn(0x7f25501551f0 0x7f255006a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- 172.21.15.33:0/1264169814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550066fb0 0x7f25500cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- 172.21.15.33:0/1264169814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550154e20 0x7f25500caa70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- 172.21.15.33:0/1264169814 >> 172.21.15.33:0/1264169814 conn(0x7f25500c8ad0 msgr2=0x7f25500c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- 172.21.15.33:0/1264169814 shutdown_connections 2023-10-06T02:45:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- 172.21.15.33:0/1264169814 wait complete. 2023-10-06T02:45:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 Processor -- start 2023-10-06T02:45:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- start start 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 0x7f255013f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550154e20 0x7f25501541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25501551f0 0x7f2550153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.617+0000 7f2556fcf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550152be0 con 0x7f2550066fb0 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2556fcf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550154730 con 0x7f25501551f0 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2556fcf700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2550151e40 con 0x7f2550154e20 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 0x7f255013f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 0x7f255013f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54986/0 (socket says 172.21.15.33:54986) 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 -- 172.21.15.33:0/2325882201 learned_addr learned my addr 172.21.15.33:0/2325882201 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f25557cc700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550154e20 0x7f25501541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f25567ce700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25501551f0 0x7f2550153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 -- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550154e20 msgr2=0x7f25501541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550154e20 0x7f25501541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 -- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25501551f0 msgr2=0x7f2550153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25501551f0 0x7f2550153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 -- 172.21.15.33:0/2325882201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c00abf0 con 0x7f2550066fb0 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2555fcd700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 0x7f255013f510 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f254c00a990 tx=0x7f254c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c008930 con 0x7f2550066fb0 2023-10-06T02:45:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c008ab0 con 0x7f2550066fb0 2023-10-06T02:45:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c008e70 con 0x7f2550066fb0 2023-10-06T02:45:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25501522b0 con 0x7f2550066fb0 2023-10-06T02:45:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2550152870 con 0x7f2550066fb0 2023-10-06T02:45:47.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.621+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2550071040 con 0x7f2550066fb0 2023-10-06T02:45:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.625+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f254c003040 con 0x7f2550066fb0 2023-10-06T02:45:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.625+0000 7f2545ffb700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2534060550 0x7f2534062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.625+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f254c07e4f0 con 0x7f2550066fb0 2023-10-06T02:45:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.625+0000 7f25557cc700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2534060550 0x7f2534062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.625+0000 7f25557cc700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2534060550 0x7f2534062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25501540d0 tx=0x7f2540008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.629+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f254c04e690 con 0x7f2550066fb0 2023-10-06T02:45:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.793+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25500c7120 con 0x7f2550066fb0 2023-10-06T02:45:47.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.793+0000 7f2545ffb700 1 -- 172.21.15.33:0/2325882201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f254c051cb0 con 0x7f2550066fb0 2023-10-06T02:45:47.800 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2534060550 msgr2=0x7f2534062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2534060550 0x7f2534062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25501540d0 tx=0x7f2540008040 comp rx=0 tx=0).stop 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 msgr2=0x7f255013f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 0x7f255013f510 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f254c00a990 tx=0x7f254c019000 comp rx=0 tx=0).stop 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 shutdown_connections 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2534060550 0x7f2534062a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2550154e20 0x7f25501541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25501551f0 0x7f2550153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 --2- 172.21.15.33:0/2325882201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2550066fb0 0x7f255013f510 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 >> 172.21.15.33:0/2325882201 conn(0x7f25500c8ad0 msgr2=0x7f25500c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 shutdown_connections 2023-10-06T02:45:47.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.797+0000 7f2556fcf700 1 -- 172.21.15.33:0/2325882201 wait complete. 2023-10-06T02:45:47.814 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:45:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 Processor -- start 2023-10-06T02:45:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 -- start start 2023-10-06T02:45:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8400158bf0 0x7f84000b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8400158fc0 0x7f840015d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840015d830 con 0x7f84000b0530 2023-10-06T02:45:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8400067c10 con 0x7f8400158bf0 2023-10-06T02:45:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8407aeb700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84000b00a0 con 0x7f8400158fc0 2023-10-06T02:45:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405887700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8400158bf0 0x7f84000b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405086700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f84000b0530 0x7f840015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44830/0 (socket says 172.21.15.33:44830) 2023-10-06T02:45:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405086700 1 -- 172.21.15.33:0/2734811366 learned_addr learned my addr 172.21.15.33:0/2734811366 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8406088700 1 --2- 172.21.15.33:0/2734811366 >> v2:172.21.15.167:3300/0 conn(0x7f8400158fc0 0x7f840015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405887700 1 -- 172.21.15.33:0/2734811366 >> v2:172.21.15.167:3300/0 conn(0x7f8400158fc0 msgr2=0x7f840015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:47.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405887700 1 --2- 172.21.15.33:0/2734811366 >> v2:172.21.15.167:3300/0 conn(0x7f8400158fc0 0x7f840015d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405887700 1 -- 172.21.15.33:0/2734811366 >> v1:172.21.15.33:6789/0 conn(0x7f84000b0530 legacy=0x7f840015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8404885700 1 -- 172.21.15.33:0/2734811366 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448235599 0 0) 0x7f840015d830 con 0x7f84000b0530 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405887700 1 -- 172.21.15.33:0/2734811366 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000cba20 con 0x7f8400158bf0 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8406088700 1 --2- 172.21.15.33:0/2734811366 >> v2:172.21.15.167:3300/0 conn(0x7f8400158fc0 0x7f840015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.925+0000 7f8405887700 1 --2- 172.21.15.33:0/2734811366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000b1b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84000ca2c0 tx=0x7f83f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1222a32af7bd393 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8404885700 1 -- 172.21.15.33:0/2734811366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4011630 con 0x7f8400158bf0 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8404885700 1 -- 172.21.15.33:0/2734811366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4011790 con 0x7f8400158bf0 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8404885700 1 -- 172.21.15.33:0/2734811366 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4011b30 con 0x7f8400158bf0 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 -- 172.21.15.33:0/2734811366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 msgr2=0x7f84000b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 --2- 172.21.15.33:0/2734811366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000b1b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84000ca2c0 tx=0x7f83f40091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 -- 172.21.15.33:0/2734811366 shutdown_connections 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 --2- 172.21.15.33:0/2734811366 >> v2:172.21.15.167:3300/0 conn(0x7f8400158fc0 0x7f840015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 --2- 172.21.15.33:0/2734811366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000b1b90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 -- 172.21.15.33:0/2734811366 >> 172.21.15.33:0/2734811366 conn(0x7f84000cfd70 msgr2=0x7f84000d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:47.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 -- 172.21.15.33:0/2734811366 shutdown_connections 2023-10-06T02:45:47.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 -- 172.21.15.33:0/2734811366 wait complete. 2023-10-06T02:45:47.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 Processor -- start 2023-10-06T02:45:47.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.929+0000 7f8407aeb700 1 -- start start 2023-10-06T02:45:47.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8400158fc0 0x7f84001552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84000c41a0 0x7f840015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c4610 con 0x7f8400158bf0 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c48f0 con 0x7f84000c41a0 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c4bd0 con 0x7f8400158fc0 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54998/0 (socket says 172.21.15.33:54998) 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 -- 172.21.15.33:0/3850471937 learned_addr learned my addr 172.21.15.33:0/3850471937 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8406088700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84000c41a0 0x7f840015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405086700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8400158fc0 0x7f84001552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 -- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8400158fc0 msgr2=0x7f84001552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8400158fc0 0x7f84001552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 -- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84000c41a0 msgr2=0x7f840015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:47.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84000c41a0 0x7f840015baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:47.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 -- 172.21.15.33:0/3850471937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8009a20 con 0x7f8400158bf0 2023-10-06T02:45:47.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8405887700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000c56b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84000694e0 tx=0x7f83f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4011b10 con 0x7f8400158bf0 2023-10-06T02:45:47.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f40233f0 con 0x7f8400158bf0 2023-10-06T02:45:47.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4023790 con 0x7f8400158bf0 2023-10-06T02:45:47.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4009420 con 0x7f8400158bf0 2023-10-06T02:45:47.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.933+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84000c5190 con 0x7f8400158bf0 2023-10-06T02:45:47.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.937+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84000710b0 con 0x7f8400158bf0 2023-10-06T02:45:47.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.937+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f83f4023b80 con 0x7f8400158bf0 2023-10-06T02:45:47.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.937+0000 7f83f1ffb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83e4060550 0x7f83e4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:47.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.937+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f83f407de80 con 0x7f8400158bf0 2023-10-06T02:45:47.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.937+0000 7f8405086700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83e4060550 0x7f83e4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:47.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.937+0000 7f8405086700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83e4060550 0x7f83e4062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84000c9f80 tx=0x7f83fc007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:47.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:47.945+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f83f404e160 con 0x7f8400158bf0 2023-10-06T02:45:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f84000ca410 con 0x7f8400158bf0 2023-10-06T02:45:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f83f1ffb700 1 -- 172.21.15.33:0/3850471937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f83f40150a0 con 0x7f8400158bf0 2023-10-06T02:45:48.106 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:45:48.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83e4060550 msgr2=0x7f83e4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83e4060550 0x7f83e4062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84000c9f80 tx=0x7f83fc007f50 comp rx=0 tx=0).stop 2023-10-06T02:45:48.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 msgr2=0x7f84000c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000c56b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84000694e0 tx=0x7f83f4017000 comp rx=0 tx=0).stop 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 shutdown_connections 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f83e4060550 0x7f83e4062a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8400158fc0 0x7f84001552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84000c41a0 0x7f840015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 --2- 172.21.15.33:0/3850471937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8400158bf0 0x7f84000c56b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 >> 172.21.15.33:0/3850471937 conn(0x7f84000cfd70 msgr2=0x7f84000cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 shutdown_connections 2023-10-06T02:45:48.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.101+0000 7f8407aeb700 1 -- 172.21.15.33:0/3850471937 wait complete. 2023-10-06T02:45:48.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:45:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.233+0000 7fdb4284e700 1 Processor -- start 2023-10-06T02:45:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.233+0000 7fdb4284e700 1 -- start start 2023-10-06T02:45:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4284e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdb3c0c9250 0x7fdb3c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4284e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3c0b07a0 con 0x7fdb3c156d20 2023-10-06T02:45:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4284e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3c0b0d60 con 0x7fdb3c0cc510 2023-10-06T02:45:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4284e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c1570f0 con 0x7fdb3c0c9250 2023-10-06T02:45:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4184c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdb3c156d20 0x7fdb3c0ce200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44832/0 (socket says 172.21.15.33:44832) 2023-10-06T02:45:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4184c700 1 -- 172.21.15.33:0/1701737174 learned_addr learned my addr 172.21.15.33:0/1701737174 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4204d700 1 --2- 172.21.15.33:0/1701737174 >> v2:172.21.15.167:3300/0 conn(0x7fdb3c0c9250 0x7fdb3c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531222813 0 0) 0x7fdb3c0b07a0 con 0x7fdb3c156d20 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb28003570 con 0x7fdb3c156d20 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283651645 0 0) 0x7fdb3c0b0d60 con 0x7fdb3c0cc510 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb3c0b07a0 con 0x7fdb3c0cc510 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4204d700 1 -- 172.21.15.33:0/1701737174 >> v1:172.21.15.167:6789/0 conn(0x7fdb3c0cc510 legacy=0x7fdb3c0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4204d700 1 -- 172.21.15.33:0/1701737174 >> v1:172.21.15.33:6789/0 conn(0x7fdb3c156d20 legacy=0x7fdb3c0ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4204d700 1 -- 172.21.15.33:0/1701737174 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c0cca10 con 0x7fdb3c0c9250 2023-10-06T02:45:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4204d700 1 --2- 172.21.15.33:0/1701737174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c161820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009e60 tx=0x7fdb34009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a317a2953d09f28d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400e070 con 0x7fdb3c0c9250 2023-10-06T02:45:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3400bb40 con 0x7fdb3c0c9250 2023-10-06T02:45:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.237+0000 7fdb4084a700 1 -- 172.21.15.33:0/1701737174 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34012430 con 0x7fdb3c0c9250 2023-10-06T02:45:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 -- 172.21.15.33:0/1701737174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 msgr2=0x7fdb3c161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 --2- 172.21.15.33:0/1701737174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c161820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009e60 tx=0x7fdb34009780 comp rx=0 tx=0).stop 2023-10-06T02:45:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 -- 172.21.15.33:0/1701737174 shutdown_connections 2023-10-06T02:45:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 --2- 172.21.15.33:0/1701737174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c161820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 -- 172.21.15.33:0/1701737174 >> 172.21.15.33:0/1701737174 conn(0x7fdb3c06aab0 msgr2=0x7fdb3c0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 -- 172.21.15.33:0/1701737174 shutdown_connections 2023-10-06T02:45:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 -- 172.21.15.33:0/1701737174 wait complete. 2023-10-06T02:45:48.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 Processor -- start 2023-10-06T02:45:48.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.241+0000 7fdb4284e700 1 -- start start 2023-10-06T02:45:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb3c0cc510 0x7fdb3c148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 0x7fdb3c148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0c6180 con 0x7fdb3c156d20 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0c4780 con 0x7fdb3c0c9250 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0c5850 con 0x7fdb3c0cc510 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 0x7fdb3c148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 0x7fdb3c148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55004/0 (socket says 172.21.15.33:55004) 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 -- 172.21.15.33:0/2304380557 learned_addr learned my addr 172.21.15.33:0/2304380557 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4104b700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb3c0cc510 0x7fdb3c148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4184c700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.250 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 -- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb3c0cc510 msgr2=0x7fdb3c148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb3c0cc510 0x7fdb3c148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 -- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 msgr2=0x7fdb3c0c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c0c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 -- 172.21.15.33:0/2304380557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34009a20 con 0x7fdb3c156d20 2023-10-06T02:45:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4184c700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c0c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:48.251 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4204d700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 0x7fdb3c148fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb34013000 tx=0x7fdb34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34026480 con 0x7fdb3c156d20 2023-10-06T02:45:48.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c0c5b30 con 0x7fdb3c156d20 2023-10-06T02:45:48.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34026600 con 0x7fdb3c156d20 2023-10-06T02:45:48.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340269c0 con 0x7fdb3c156d20 2023-10-06T02:45:48.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.245+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c0c2030 con 0x7fdb3c156d20 2023-10-06T02:45:48.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.249+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c0c62f0 con 0x7fdb3c156d20 2023-10-06T02:45:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.249+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdb3401c400 con 0x7fdb3c156d20 2023-10-06T02:45:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.249+0000 7fdb31ffb700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb20060550 0x7fdb20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.253 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.249+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdb3407e900 con 0x7fdb3c156d20 2023-10-06T02:45:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.249+0000 7fdb4184c700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb20060550 0x7fdb20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.249+0000 7fdb4184c700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb20060550 0x7fdb20062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb38004c50 tx=0x7fdb38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.261+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdb3404ea00 con 0x7fdb3c156d20 2023-10-06T02:45:48.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.417+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb3c071040 con 0x7fdb3c156d20 2023-10-06T02:45:48.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.417+0000 7fdb31ffb700 1 -- 172.21.15.33:0/2304380557 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb34052020 con 0x7fdb3c156d20 2023-10-06T02:45:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb20060550 msgr2=0x7fdb20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb20060550 0x7fdb20062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb38004c50 tx=0x7fdb38009040 comp rx=0 tx=0).stop 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 msgr2=0x7fdb3c148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 0x7fdb3c148fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb34013000 tx=0x7fdb34019000 comp rx=0 tx=0).stop 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 shutdown_connections 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdb20060550 0x7fdb20062a00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb3c0cc510 0x7fdb3c148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb3c156d20 0x7fdb3c148fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 --2- 172.21.15.33:0/2304380557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb3c0c9250 0x7fdb3c0c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 >> 172.21.15.33:0/2304380557 conn(0x7fdb3c06aab0 msgr2=0x7fdb3c0cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 shutdown_connections 2023-10-06T02:45:48.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.421+0000 7fdb4284e700 1 -- 172.21.15.33:0/2304380557 wait complete. 2023-10-06T02:45:48.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:45:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 Processor -- start 2023-10-06T02:45:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 -- start start 2023-10-06T02:45:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e80b0530 0x7f89e815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f89e8158fc0 0x7f89e815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e815d830 con 0x7f89e8158bf0 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e8067c10 con 0x7f89e8158fc0 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eef27700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80b00a0 con 0x7f89e80b0530 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eccc3700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f89e8158bf0 0x7f89e80b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44844/0 (socket says 172.21.15.33:44844) 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89eccc3700 1 -- 172.21.15.33:0/3912438905 learned_addr learned my addr 172.21.15.33:0/3912438905 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89ed4c4700 1 --2- 172.21.15.33:0/3912438905 >> v2:172.21.15.167:3300/0 conn(0x7f89e8158fc0 0x7f89e815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89e7fff700 1 --2- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e80b0530 0x7f89e815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89e77fe700 1 -- 172.21.15.33:0/3912438905 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933794324 0 0) 0x7f89e815d830 con 0x7f89e8158bf0 2023-10-06T02:45:48.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.553+0000 7f89e77fe700 1 -- 172.21.15.33:0/3912438905 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d8003570 con 0x7f89e8158bf0 2023-10-06T02:45:48.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89ed4c4700 1 -- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e80b0530 msgr2=0x7f89e815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89ed4c4700 1 --2- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e80b0530 0x7f89e815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89e77fe700 1 -- 172.21.15.33:0/3912438905 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671863335 0 0) 0x7f89d8003570 con 0x7f89e8158bf0 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89ed4c4700 1 -- 172.21.15.33:0/3912438905 >> v1:172.21.15.33:6789/0 conn(0x7f89e8158bf0 legacy=0x7f89e80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89ed4c4700 1 -- 172.21.15.33:0/3912438905 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80cba20 con 0x7f89e8158fc0 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89ed4c4700 1 --2- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815cfb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e60 tx=0x7f89e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdfc5c398e2726d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89e77fe700 1 -- 172.21.15.33:0/3912438905 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000e070 con 0x7f89e8158fc0 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89e77fe700 1 -- 172.21.15.33:0/3912438905 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000b7a0 con 0x7f89e8158fc0 2023-10-06T02:45:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89e77fe700 1 -- 172.21.15.33:0/3912438905 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc001470 con 0x7f89e8158fc0 2023-10-06T02:45:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 -- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 msgr2=0x7f89e815cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 --2- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815cfb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e60 tx=0x7f89e0009780 comp rx=0 tx=0).stop 2023-10-06T02:45:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 -- 172.21.15.33:0/3912438905 shutdown_connections 2023-10-06T02:45:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 --2- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815cfb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 --2- 172.21.15.33:0/3912438905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e80b0530 0x7f89e815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 -- 172.21.15.33:0/3912438905 >> 172.21.15.33:0/3912438905 conn(0x7f89e80cfd70 msgr2=0x7f89e80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:48.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 -- 172.21.15.33:0/3912438905 shutdown_connections 2023-10-06T02:45:48.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.557+0000 7f89eef27700 1 -- 172.21.15.33:0/3912438905 wait complete. 2023-10-06T02:45:48.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 Processor -- start 2023-10-06T02:45:48.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 -- start start 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e8158bf0 0x7f89e80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 0x7f89e815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e815bd50 con 0x7f89e8155b40 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80af200 con 0x7f89e8158fc0 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e815a130 con 0x7f89e8158bf0 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89ed4c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 0x7f89e815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eccc3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e8158bf0 0x7f89e80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44090/0 (socket says 172.21.15.33:44090) 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 -- 172.21.15.33:0/3647725351 learned_addr learned my addr 172.21.15.33:0/3647725351 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89ed4c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 0x7f89e815ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55016/0 (socket says 172.21.15.33:55016) 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 -- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e8158bf0 msgr2=0x7f89e80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e8158bf0 0x7f89e80aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 -- 172.21.15.33:0/3647725351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 msgr2=0x7f89e815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 0x7f89e815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 -- 172.21.15.33:0/3647725351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e0009a20 con 0x7f89e8158fc0 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89ed4c4700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 0x7f89e815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e7fff700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815b810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e930 tx=0x7f89d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4013680 con 0x7f89e8158fc0 2023-10-06T02:45:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d40137e0 con 0x7f89e8158fc0 2023-10-06T02:45:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e815a410 con 0x7f89e8158fc0 2023-10-06T02:45:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4013b80 con 0x7f89e8158fc0 2023-10-06T02:45:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.561+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e8159e20 con 0x7f89e8158fc0 2023-10-06T02:45:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.565+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e815a8f0 con 0x7f89e8158fc0 2023-10-06T02:45:48.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.565+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f89d400c3e0 con 0x7f89e8158fc0 2023-10-06T02:45:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.565+0000 7f89e4ff9700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89cc060550 0x7f89cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.565+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f89d407cd30 con 0x7f89e8158fc0 2023-10-06T02:45:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.565+0000 7f89eccc3700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89cc060550 0x7f89cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.569+0000 7f89eccc3700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89cc060550 0x7f89cc062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0011e0 tx=0x7f89dc001aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.577+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f89d404ced0 con 0x7f89e8158fc0 2023-10-06T02:45:48.743 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.737+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89e80710b0 con 0x7f89e8158fc0 2023-10-06T02:45:48.744 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.737+0000 7f89e4ff9700 1 -- 172.21.15.33:0/3647725351 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89d40504f0 con 0x7f89e8158fc0 2023-10-06T02:45:48.744 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:45:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89cc060550 msgr2=0x7f89cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89cc060550 0x7f89cc062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0011e0 tx=0x7f89dc001aa0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 msgr2=0x7f89e815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.746 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815b810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e930 tx=0x7f89d40091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 shutdown_connections 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e8155b40 0x7f89e815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89e8158fc0 0x7f89e815b810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89cc060550 0x7f89cc062a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 --2- 172.21.15.33:0/3647725351 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89e8158bf0 0x7f89e80aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 >> 172.21.15.33:0/3647725351 conn(0x7f89e80cfd70 msgr2=0x7f89e80cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 shutdown_connections 2023-10-06T02:45:48.747 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.741+0000 7f89eef27700 1 -- 172.21.15.33:0/3647725351 wait complete. 2023-10-06T02:45:48.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:45:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.865+0000 7f769a5da700 1 Processor -- start 2023-10-06T02:45:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f769a5da700 1 -- start start 2023-10-06T02:45:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f769a5da700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7694068aa0 0x7f76940b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f769a5da700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76940ce580 con 0x7f76940cf7c0 2023-10-06T02:45:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f769a5da700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76940ce860 con 0x7f7694158c60 2023-10-06T02:45:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f769a5da700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940cf620 con 0x7f7694068aa0 2023-10-06T02:45:48.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7698dd7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7694068aa0 0x7f76940b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7699dd9700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f76940cf7c0 0x7f76940b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44848/0 (socket says 172.21.15.33:44848) 2023-10-06T02:45:48.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7699dd9700 1 -- 172.21.15.33:0/987668220 learned_addr learned my addr 172.21.15.33:0/987668220 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7698dd7700 1 -- 172.21.15.33:0/987668220 >> v1:172.21.15.167:6789/0 conn(0x7f7694158c60 legacy=0x7f7694068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f768bfff700 1 -- 172.21.15.33:0/987668220 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374585975 0 0) 0x7f76940ce860 con 0x7f7694158c60 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7698dd7700 1 -- 172.21.15.33:0/987668220 >> v1:172.21.15.33:6789/0 conn(0x7f76940cf7c0 legacy=0x7f76940b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7698dd7700 1 -- 172.21.15.33:0/987668220 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76940ceb40 con 0x7f7694068aa0 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f7698dd7700 1 --2- 172.21.15.33:0/987668220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7694068aa0 0x7f76940b29e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76900098e0 tx=0x7f76900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b538cee074b3d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f768bfff700 1 -- 172.21.15.33:0/987668220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690009e80 con 0x7f7694068aa0 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f768bfff700 1 -- 172.21.15.33:0/987668220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769000b860 con 0x7f7694068aa0 2023-10-06T02:45:48.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.869+0000 7f768bfff700 1 -- 172.21.15.33:0/987668220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000bbe0 con 0x7f7694068aa0 2023-10-06T02:45:48.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- 172.21.15.33:0/987668220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7694068aa0 msgr2=0x7f76940b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 --2- 172.21.15.33:0/987668220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7694068aa0 0x7f76940b29e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76900098e0 tx=0x7f76900091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- 172.21.15.33:0/987668220 shutdown_connections 2023-10-06T02:45:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 --2- 172.21.15.33:0/987668220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7694068aa0 0x7f76940b29e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- 172.21.15.33:0/987668220 >> 172.21.15.33:0/987668220 conn(0x7f7694067a40 msgr2=0x7f76940b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- 172.21.15.33:0/987668220 shutdown_connections 2023-10-06T02:45:48.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- 172.21.15.33:0/987668220 wait complete. 2023-10-06T02:45:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 Processor -- start 2023-10-06T02:45:48.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- start start 2023-10-06T02:45:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7694068aa0 0x7f769414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 0x7f76940b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7694158c60 0x7f76940b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694157170 con 0x7f76940cf7c0 2023-10-06T02:45:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76940b29c0 con 0x7f7694068aa0 2023-10-06T02:45:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f769a5da700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7694156280 con 0x7f7694158c60 2023-10-06T02:45:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f7698dd7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 0x7f76940b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f7698dd7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 0x7f76940b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55042/0 (socket says 172.21.15.33:55042) 2023-10-06T02:45:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f7698dd7700 1 -- 172.21.15.33:0/3853545725 learned_addr learned my addr 172.21.15.33:0/3853545725 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f7699dd9700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7694158c60 0x7f76940b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.873+0000 7f76995d8700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7694068aa0 0x7f769414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f7698dd7700 1 -- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7694158c60 msgr2=0x7f76940b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f7698dd7700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7694158c60 0x7f76940b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f7698dd7700 1 -- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7694068aa0 msgr2=0x7f769414bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f7698dd7700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7694068aa0 0x7f769414bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:48.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f7698dd7700 1 -- 172.21.15.33:0/3853545725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009420 con 0x7f76940cf7c0 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f76995d8700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7694068aa0 0x7f769414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f7698dd7700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 0x7f76940b4cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76940696d0 tx=0x7f7690017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000bbe0 con 0x7f76940cf7c0 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7690011420 con 0x7f76940cf7c0 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900117e0 con 0x7f76940cf7c0 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7694156560 con 0x7f76940cf7c0 2023-10-06T02:45:48.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7694156b20 con 0x7f76940cf7c0 2023-10-06T02:45:48.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694071040 con 0x7f76940cf7c0 2023-10-06T02:45:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.877+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7690011bd0 con 0x7f76940cf7c0 2023-10-06T02:45:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.881+0000 7f76897fa700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7678060550 0x7f7678062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.881+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f769007e700 con 0x7f76940cf7c0 2023-10-06T02:45:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.881+0000 7f76995d8700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7678060550 0x7f7678062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:48.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.881+0000 7f76995d8700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7678060550 0x7f7678062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7684001860 tx=0x7f7684009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:48.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:48.885+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f769004e8a0 con 0x7f76940cf7c0 2023-10-06T02:45:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.049+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f76940cee20 con 0x7f76940cf7c0 2023-10-06T02:45:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.049+0000 7f76897fa700 1 -- 172.21.15.33:0/3853545725 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7690051ec0 con 0x7f76940cf7c0 2023-10-06T02:45:49.054 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.049+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7678060550 msgr2=0x7f7678062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.049+0000 7f769a5da700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7678060550 0x7f7678062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7684001860 tx=0x7f7684009040 comp rx=0 tx=0).stop 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.049+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 msgr2=0x7f76940b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.049+0000 7f769a5da700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 0x7f76940b4cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76940696d0 tx=0x7f7690017000 comp rx=0 tx=0).stop 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 shutdown_connections 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7678060550 0x7f7678062a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7694158c60 0x7f76940b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76940cf7c0 0x7f76940b4cd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 --2- 172.21.15.33:0/3853545725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7694068aa0 0x7f769414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 >> 172.21.15.33:0/3853545725 conn(0x7f7694067a40 msgr2=0x7f769414db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 shutdown_connections 2023-10-06T02:45:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.053+0000 7f769a5da700 1 -- 172.21.15.33:0/3853545725 wait complete. 2023-10-06T02:45:49.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:49.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.169+0000 7fe8f1454700 1 Processor -- start 2023-10-06T02:45:49.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.169+0000 7fe8f1454700 1 -- start start 2023-10-06T02:45:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 0x7fe8ec066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8ec0c67e0 0x7fe8ec0c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec06b420 con 0x7fe8ec06b050 2023-10-06T02:45:49.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec0674f0 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec06a4e0 con 0x7fe8ec0c93a0 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f0c53700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8ec0c67e0 0x7fe8ec0c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 0x7fe8ec066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 0x7fe8ec066fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:54938/0 (socket says 172.21.15.33:54938) 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/4141771971 learned_addr learned my addr 172.21.15.33:0/4141771971 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f0c53700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8ec0c67e0 0x7fe8ec0c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55044/0 (socket says 172.21.15.33:55044) 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8ebfff700 1 --1- 172.21.15.33:0/4141771971 >> v1:172.21.15.167:6789/0 conn(0x7fe8ec06b050 0x7fe8ec0c60d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33778/0 (socket says 172.21.15.33:33778) 2023-10-06T02:45:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/4141771971 >> v2:172.21.15.33:3300/0 conn(0x7fe8ec0c67e0 msgr2=0x7fe8ec0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 --2- 172.21.15.33:0/4141771971 >> v2:172.21.15.33:3300/0 conn(0x7fe8ec0c67e0 0x7fe8ec0c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/4141771971 >> v1:172.21.15.167:6789/0 conn(0x7fe8ec06b050 legacy=0x7fe8ec0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/4141771971 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec067700 con 0x7fe8ec0c93a0 2023-10-06T02:45:49.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eb7fe700 1 --2- 172.21.15.33:0/4141771971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 0x7fe8ec066fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40099b0 tx=0x7fe8d4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b942aee7acca1249 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eaffd700 1 -- 172.21.15.33:0/4141771971 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4009ea0 con 0x7fe8ec0c93a0 2023-10-06T02:45:49.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eaffd700 1 -- 172.21.15.33:0/4141771971 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400b860 con 0x7fe8ec0c93a0 2023-10-06T02:45:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8eaffd700 1 -- 172.21.15.33:0/4141771971 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400bc00 con 0x7fe8ec0c93a0 2023-10-06T02:45:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- 172.21.15.33:0/4141771971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 msgr2=0x7fe8ec066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 --2- 172.21.15.33:0/4141771971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 0x7fe8ec066fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40099b0 tx=0x7fe8d4009250 comp rx=0 tx=0).stop 2023-10-06T02:45:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- 172.21.15.33:0/4141771971 shutdown_connections 2023-10-06T02:45:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 --2- 172.21.15.33:0/4141771971 >> v2:172.21.15.33:3300/0 conn(0x7fe8ec0c67e0 0x7fe8ec0c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 --2- 172.21.15.33:0/4141771971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec0c93a0 0x7fe8ec066fb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- 172.21.15.33:0/4141771971 >> 172.21.15.33:0/4141771971 conn(0x7fe8ec0c8240 msgr2=0x7fe8ec0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:49.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- 172.21.15.33:0/4141771971 shutdown_connections 2023-10-06T02:45:49.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.173+0000 7fe8f1454700 1 -- 172.21.15.33:0/4141771971 wait complete. 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 Processor -- start 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 -- start start 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec06b050 0x7fe8ec145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 0x7fe8ec06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ec0c93a0 0x7fe8ec06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec145fc0 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0af7e0 con 0x7fe8ec0c93a0 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f1454700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0aeeb0 con 0x7fe8ec06b050 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 0x7fe8ec06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 0x7fe8ec06d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55056/0 (socket says 172.21.15.33:55056) 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/2875473843 learned_addr learned my addr 172.21.15.33:0/2875473843 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8f0c53700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ec0c93a0 0x7fe8ec06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.177+0000 7fe8ebfff700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec06b050 0x7fe8ec145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec06b050 msgr2=0x7fe8ec145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8eb7fe700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec06b050 0x7fe8ec145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ec0c93a0 msgr2=0x7fe8ec06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8eb7fe700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ec0c93a0 0x7fe8ec06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8eb7fe700 1 -- 172.21.15.33:0/2875473843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40094f0 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8eb7fe700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 0x7fe8ec06d830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec067630 tx=0x7fe8d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400bbe0 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d401a450 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d401a7f0 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0af190 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec15a520 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec0af950 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe8d401ab60 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8e3fff700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe8d0060550 0x7fe8d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe8d407eb60 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.181+0000 7fe8ebfff700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe8d0060550 0x7fe8d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.185+0000 7fe8ebfff700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe8d0060550 0x7fe8d0062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005f80 tx=0x7fe8dc007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.189+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe8d404ed00 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.385+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8ec0b0dd0 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.385+0000 7fe8e3fff700 1 -- 172.21.15.33:0/2875473843 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fe8d4052320 con 0x7fe8ec0c67e0 2023-10-06T02:45:49.392 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:49.392 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:45:49.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe8d0060550 msgr2=0x7fe8d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe8d0060550 0x7fe8d0062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005f80 tx=0x7fe8dc007ce0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.394 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 msgr2=0x7fe8ec06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 0x7fe8ec06d830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec067630 tx=0x7fe8d4017000 comp rx=0 tx=0).stop 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 shutdown_connections 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ec0c93a0 0x7fe8ec06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8ec0c67e0 0x7fe8ec06d830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe8d0060550 0x7fe8d0062a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 --2- 172.21.15.33:0/2875473843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ec06b050 0x7fe8ec145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 >> 172.21.15.33:0/2875473843 conn(0x7fe8ec0c8240 msgr2=0x7fe8ec0ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 shutdown_connections 2023-10-06T02:45:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.389+0000 7fe8f1454700 1 -- 172.21.15.33:0/2875473843 wait complete. 2023-10-06T02:45:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:45:49.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:49.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.509+0000 7f6844c0c700 1 Processor -- start 2023-10-06T02:45:49.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.509+0000 7f6844c0c700 1 -- start start 2023-10-06T02:45:49.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f6840163c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68400af6c0 con 0x7f684015d600 2023-10-06T02:45:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68400af910 con 0x7f6840158bc0 2023-10-06T02:45:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68400afbf0 con 0x7f6840159fa0 2023-10-06T02:45:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f684015d600 0x7f684006bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44850/0 (socket says 172.21.15.33:44850) 2023-10-06T02:45:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683effd700 1 -- 172.21.15.33:0/872218479 learned_addr learned my addr 172.21.15.33:0/872218479 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:49.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683ffff700 1 --2- 172.21.15.33:0/872218479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f6840163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916117908 0 0) 0x7f68400af6c0 con 0x7f684015d600 2023-10-06T02:45:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6828003570 con 0x7f684015d600 2023-10-06T02:45:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899346514 0 0) 0x7f68400af910 con 0x7f6840158bc0 2023-10-06T02:45:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68400af6c0 con 0x7f6840158bc0 2023-10-06T02:45:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683ffff700 1 -- 172.21.15.33:0/872218479 >> v1:172.21.15.167:6789/0 conn(0x7f6840158bc0 legacy=0x7f68400b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683ffff700 1 -- 172.21.15.33:0/872218479 >> v1:172.21.15.33:6789/0 conn(0x7f684015d600 legacy=0x7f684006bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683ffff700 1 -- 172.21.15.33:0/872218479 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400afed0 con 0x7f6840159fa0 2023-10-06T02:45:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683ffff700 1 --2- 172.21.15.33:0/872218479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f6840163c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6838009e90 tx=0x7f6838009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca59634bbba9b65f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800e070 con 0x7f6840159fa0 2023-10-06T02:45:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683800bb80 con 0x7f6840159fa0 2023-10-06T02:45:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f683e7fc700 1 -- 172.21.15.33:0/872218479 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838012430 con 0x7f6840159fa0 2023-10-06T02:45:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- 172.21.15.33:0/872218479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 msgr2=0x7f6840163c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 --2- 172.21.15.33:0/872218479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f6840163c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6838009e90 tx=0x7f6838009780 comp rx=0 tx=0).stop 2023-10-06T02:45:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- 172.21.15.33:0/872218479 shutdown_connections 2023-10-06T02:45:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 --2- 172.21.15.33:0/872218479 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f6840163c50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- 172.21.15.33:0/872218479 >> 172.21.15.33:0/872218479 conn(0x7f684006e6c0 msgr2=0x7f684006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:49.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- 172.21.15.33:0/872218479 shutdown_connections 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.513+0000 7f6844c0c700 1 -- 172.21.15.33:0/872218479 wait complete. 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 Processor -- start 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 -- start start 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 0x7f684014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f684014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f684014dd00 0x7f684014e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684014ca60 con 0x7f6840158bc0 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684014e6c0 con 0x7f684014dd00 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f6844c0c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684014ebc0 con 0x7f6840159fa0 2023-10-06T02:45:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 0x7f684014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 0x7f684014c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55064/0 (socket says 172.21.15.33:55064) 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 -- 172.21.15.33:0/2348598716 learned_addr learned my addr 172.21.15.33:0/2348598716 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683effd700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f684014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683ffff700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f684014dd00 0x7f684014e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 -- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 msgr2=0x7f684014fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f684014fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 -- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f684014dd00 msgr2=0x7f684014e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f684014dd00 0x7f684014e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 -- 172.21.15.33:0/2348598716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838009a20 con 0x7f6840158bc0 2023-10-06T02:45:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.517+0000 7f683f7fe700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 0x7f684014c520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f683000c7c0 tx=0x7f6830007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830001530 con 0x7f6840158bc0 2023-10-06T02:45:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68300016b0 con 0x7f6840158bc0 2023-10-06T02:45:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684014d480 con 0x7f6840158bc0 2023-10-06T02:45:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830001a70 con 0x7f6840158bc0 2023-10-06T02:45:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6840157f30 con 0x7f6840158bc0 2023-10-06T02:45:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684015cc70 con 0x7f6840158bc0 2023-10-06T02:45:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f683001b3b0 con 0x7f6840158bc0 2023-10-06T02:45:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f68277fe700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f682c060550 0x7f682c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.521+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f683007c3d0 con 0x7f6840158bc0 2023-10-06T02:45:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.525+0000 7f683effd700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f682c060550 0x7f682c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.525+0000 7f683effd700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f682c060550 0x7f682c062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f684016c930 tx=0x7f6834009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.533+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f684015cc70 con 0x7f6840158bc0 2023-10-06T02:45:49.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.693+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f684015cc70 con 0x7f6840158bc0 2023-10-06T02:45:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f68277fe700 1 -- 172.21.15.33:0/2348598716 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f684015cc70 con 0x7f6840158bc0 2023-10-06T02:45:49.702 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:49.702 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f682c060550 msgr2=0x7f682c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f682c060550 0x7f682c062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f684016c930 tx=0x7f6834009040 comp rx=0 tx=0).stop 2023-10-06T02:45:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 msgr2=0x7f684014c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 0x7f684014c520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f683000c7c0 tx=0x7f6830007dc0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 shutdown_connections 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f682c060550 0x7f682c062a00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6840159fa0 0x7f684014fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f684014dd00 0x7f684014e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 --2- 172.21.15.33:0/2348598716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840158bc0 0x7f684014c520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 >> 172.21.15.33:0/2348598716 conn(0x7f684006e6c0 msgr2=0x7f6840068110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.697+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 shutdown_connections 2023-10-06T02:45:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.701+0000 7f6844c0c700 1 -- 172.21.15.33:0/2348598716 wait complete. 2023-10-06T02:45:49.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:45:49.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:45:49.755 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:45:49.758 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:45:49.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b6be9700 1 Processor -- start 2023-10-06T02:45:49.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b6be9700 1 -- start start 2023-10-06T02:45:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b6be9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09b00c7840 0x7f09b006c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b6be9700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b0068ed0 con 0x7f09b00cc3d0 2023-10-06T02:45:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b6be9700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00682c0 con 0x7f09b00cc000 2023-10-06T02:45:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b6be9700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b01591f0 con 0x7f09b00c7840 2023-10-06T02:45:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b53e6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f09b00c7840 0x7f09b006c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b63e8700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f09b00cc3d0 0x7f09b00ca4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44852/0 (socket says 172.21.15.33:44852) 2023-10-06T02:45:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b63e8700 1 -- 172.21.15.33:0/1860373792 learned_addr learned my addr 172.21.15.33:0/1860373792 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b53e6700 1 -- 172.21.15.33:0/1860373792 >> v1:172.21.15.167:6789/0 conn(0x7f09b00cc000 legacy=0x7f09b00c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b53e6700 1 -- 172.21.15.33:0/1860373792 >> v1:172.21.15.33:6789/0 conn(0x7f09b00cc3d0 legacy=0x7f09b00ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b53e6700 1 -- 172.21.15.33:0/1860373792 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00685a0 con 0x7f09b00c7840 2023-10-06T02:45:49.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.829+0000 7f09b53e6700 1 --2- 172.21.15.33:0/1860373792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7840 0x7f09b006c3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f09a00098e0 tx=0x7f09a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43eef608eb2fd98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b4be5700 1 -- 172.21.15.33:0/1860373792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0009e80 con 0x7f09b00c7840 2023-10-06T02:45:49.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b4be5700 1 -- 172.21.15.33:0/1860373792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a000b860 con 0x7f09b00c7840 2023-10-06T02:45:49.837 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b4be5700 1 -- 172.21.15.33:0/1860373792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000bbe0 con 0x7f09b00c7840 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 -- 172.21.15.33:0/1860373792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7840 msgr2=0x7f09b006c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 --2- 172.21.15.33:0/1860373792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7840 0x7f09b006c3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f09a00098e0 tx=0x7f09a00091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 -- 172.21.15.33:0/1860373792 shutdown_connections 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 --2- 172.21.15.33:0/1860373792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7840 0x7f09b006c3d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 -- 172.21.15.33:0/1860373792 >> 172.21.15.33:0/1860373792 conn(0x7f09b00c9350 msgr2=0x7f09b00c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 -- 172.21.15.33:0/1860373792 shutdown_connections 2023-10-06T02:45:49.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 -- 172.21.15.33:0/1860373792 wait complete. 2023-10-06T02:45:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.833+0000 7f09b6be9700 1 Processor -- start 2023-10-06T02:45:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 -- start start 2023-10-06T02:45:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b00c7840 0x7f09b00c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09b00cc000 0x7f09b00c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 0x7f09b0147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00c3ed0 con 0x7f09b00cc3d0 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00c4d10 con 0x7f09b00c7840 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b01478b0 con 0x7f09b00cc000 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 0x7f09b0147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 0x7f09b0147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55096/0 (socket says 172.21.15.33:55096) 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 -- 172.21.15.33:0/2615889513 learned_addr learned my addr 172.21.15.33:0/2615889513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b53e6700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09b00cc000 0x7f09b00c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b5be7700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b00c7840 0x7f09b00c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 -- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09b00cc000 msgr2=0x7f09b00c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09b00cc000 0x7f09b00c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 -- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b00c7840 msgr2=0x7f09b00c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b00c7840 0x7f09b00c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 -- 172.21.15.33:0/2615889513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0009420 con 0x7f09b00cc3d0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b63e8700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 0x7f09b0147370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cc40 tx=0x7f09a8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800a8b0 con 0x7f09b00cc3d0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800aa10 con 0x7f09b00cc3d0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b00c4100 con 0x7f09b00cc3d0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800add0 con 0x7f09b00cc3d0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.837+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b00c46c0 con 0x7f09b00cc3d0 2023-10-06T02:45:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.841+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b00c4e80 con 0x7f09b00cc3d0 2023-10-06T02:45:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.841+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f09a8007300 con 0x7f09b00cc3d0 2023-10-06T02:45:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.841+0000 7f09a5ffb700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0994060550 0x7f0994062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.841+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f09a807cf10 con 0x7f09b00cc3d0 2023-10-06T02:45:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.841+0000 7f09b5be7700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0994060550 0x7f0994062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.845+0000 7f09b5be7700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0994060550 0x7f0994062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09ac007fd0 tx=0x7f09ac007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:49.849+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f09a804d130 con 0x7f09b00cc3d0 2023-10-06T02:45:50.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.013+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09b0071040 con 0x7f09b00cc3d0 2023-10-06T02:45:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09a5ffb700 1 -- 172.21.15.33:0/2615889513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09a8050750 con 0x7f09b00cc3d0 2023-10-06T02:45:50.021 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:45:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0994060550 msgr2=0x7f0994062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0994060550 0x7f0994062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09ac007fd0 tx=0x7f09ac007eb0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 msgr2=0x7f09b0147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 0x7f09b0147370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cc40 tx=0x7f09a8001a50 comp rx=0 tx=0).stop 2023-10-06T02:45:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 shutdown_connections 2023-10-06T02:45:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0994060550 0x7f0994062a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09b00cc000 0x7f09b00c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00cc3d0 0x7f09b0147370 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 --2- 172.21.15.33:0/2615889513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b00c7840 0x7f09b00c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 >> 172.21.15.33:0/2615889513 conn(0x7f09b00c9350 msgr2=0x7f09b01563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.017+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 shutdown_connections 2023-10-06T02:45:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.021+0000 7f09b6be9700 1 -- 172.21.15.33:0/2615889513 wait complete. 2023-10-06T02:45:50.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:45:50.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.145+0000 7f9cfd361700 1 Processor -- start 2023-10-06T02:45:50.152 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.145+0000 7f9cfd361700 1 -- start start 2023-10-06T02:45:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cf8156bd0 0x7f9cf80ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80af1e0 con 0x7f9cf80ae500 2023-10-06T02:45:50.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80b0890 con 0x7f9cf8156bd0 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8067450 con 0x7f9cf80abd60 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cf8156bd0 0x7f9cf80ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9cf8156bd0 0x7f9cf80ab820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55100/0 (socket says 172.21.15.33:55100) 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 -- 172.21.15.33:0/3892659305 learned_addr learned my addr 172.21.15.33:0/3892659305 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf77fe700 1 --1- 172.21.15.33:0/3892659305 >> v1:172.21.15.33:6789/0 conn(0x7f9cf80ae500 0x7f9cf80b1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44868/0 (socket says 172.21.15.33:44868) 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf67fc700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 -- 172.21.15.33:0/3892659305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 msgr2=0x7f9cf80adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf80adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 -- 172.21.15.33:0/3892659305 >> v1:172.21.15.33:6789/0 conn(0x7f9cf80ae500 legacy=0x7f9cf80b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 -- 172.21.15.33:0/3892659305 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80ac1b0 con 0x7f9cf8156bd0 2023-10-06T02:45:50.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf67fc700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:50.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf6ffd700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf8156bd0 0x7f9cf80ab820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00099b0 tx=0x7f9ce0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789c1c313f8f3ec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf5ffb700 1 -- 172.21.15.33:0/3892659305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000b860 con 0x7f9cf8156bd0 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf5ffb700 1 -- 172.21.15.33:0/3892659305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000b9c0 con 0x7f9cf8156bd0 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cf5ffb700 1 -- 172.21.15.33:0/3892659305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000bd60 con 0x7f9cf8156bd0 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- 172.21.15.33:0/3892659305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf8156bd0 msgr2=0x7f9cf80ab820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf8156bd0 0x7f9cf80ab820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00099b0 tx=0x7f9ce0009250 comp rx=0 tx=0).stop 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- 172.21.15.33:0/3892659305 shutdown_connections 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 --2- 172.21.15.33:0/3892659305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf8156bd0 0x7f9cf80ab820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.156 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- 172.21.15.33:0/3892659305 >> 172.21.15.33:0/3892659305 conn(0x7f9cf80af9c0 msgr2=0x7f9cf80afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- 172.21.15.33:0/3892659305 shutdown_connections 2023-10-06T02:45:50.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.149+0000 7f9cfd361700 1 -- 172.21.15.33:0/3892659305 wait complete. 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 Processor -- start 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- start start 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf809a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 0x7f9cf815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8156bd0 0x7f9cf816c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80ab070 con 0x7f9cf80ae500 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf809a540 con 0x7f9cf8156bd0 2023-10-06T02:45:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8099b70 con 0x7f9cf80abd60 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 0x7f9cf815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 0x7f9cf815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55112/0 (socket says 172.21.15.33:55112) 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 -- 172.21.15.33:0/559062928 learned_addr learned my addr 172.21.15.33:0/559062928 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf6ffd700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf809a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf77fe700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8156bd0 0x7f9cf816c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 -- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 msgr2=0x7f9cf809a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf809a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 -- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8156bd0 msgr2=0x7f9cf816c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8156bd0 0x7f9cf816c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 -- 172.21.15.33:0/559062928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00094f0 con 0x7f9cf80ae500 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cf67fc700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 0x7f9cf815a3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800e900 tx=0x7f9ce8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800eea0 con 0x7f9cf80ae500 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800c5e0 con 0x7f9cf80ae500 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf8098fc0 con 0x7f9cf80ae500 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800c980 con 0x7f9cf80ae500 2023-10-06T02:45:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf8099800 con 0x7f9cf80ae500 2023-10-06T02:45:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.153+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf8099d00 con 0x7f9cf80ae500 2023-10-06T02:45:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.157+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9ce8025430 con 0x7f9cf80ae500 2023-10-06T02:45:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.157+0000 7f9ceeffd700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9cd8060550 0x7f9cd8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.157+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9ce807cda0 con 0x7f9cf80ae500 2023-10-06T02:45:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.157+0000 7f9cf6ffd700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9cd8060550 0x7f9cd8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.157+0000 7f9cf6ffd700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9cd8060550 0x7f9cd8062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00099b0 tx=0x7f9ce0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.161+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9ce804cfc0 con 0x7f9cf80ae500 2023-10-06T02:45:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.317+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9cf80710b0 con 0x7f9cf80ae500 2023-10-06T02:45:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.317+0000 7f9ceeffd700 1 -- 172.21.15.33:0/559062928 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ce8020020 con 0x7f9cf80ae500 2023-10-06T02:45:50.323 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:45:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9cd8060550 msgr2=0x7f9cd8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9cd8060550 0x7f9cd8062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00099b0 tx=0x7f9ce0017000 comp rx=0 tx=0).stop 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 msgr2=0x7f9cf815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 0x7f9cf815a3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800e900 tx=0x7f9ce8008040 comp rx=0 tx=0).stop 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 shutdown_connections 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf8156bd0 0x7f9cf816c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf80ae500 0x7f9cf815a3d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9cd8060550 0x7f9cd8062a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 --2- 172.21.15.33:0/559062928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf80abd60 0x7f9cf809a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 >> 172.21.15.33:0/559062928 conn(0x7f9cf80af9c0 msgr2=0x7f9cf806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 shutdown_connections 2023-10-06T02:45:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.321+0000 7f9cfd361700 1 -- 172.21.15.33:0/559062928 wait complete. 2023-10-06T02:45:50.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:45:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.441+0000 7f04c359e700 1 Processor -- start 2023-10-06T02:45:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.441+0000 7f04c359e700 1 -- start start 2023-10-06T02:45:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f04c4156860 0x7f04c40c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04c415acd0 0x7f04c4069de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c406bb80 con 0x7f04c4156c30 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c40ae5e0 con 0x7f04c415acd0 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c40ae8c0 con 0x7f04c4156860 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04c415acd0 0x7f04c4069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04c415acd0 0x7f04c4069de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55116/0 (socket says 172.21.15.33:55116) 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 -- 172.21.15.33:0/3914935925 learned_addr learned my addr 172.21.15.33:0/3914935925 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c259c700 1 --2- 172.21.15.33:0/3914935925 >> v2:172.21.15.167:3300/0 conn(0x7f04c4156860 0x7f04c40c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 -- 172.21.15.33:0/3914935925 >> v2:172.21.15.167:3300/0 conn(0x7f04c4156860 msgr2=0x7f04c40c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 --2- 172.21.15.33:0/3914935925 >> v2:172.21.15.167:3300/0 conn(0x7f04c4156860 0x7f04c40c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 -- 172.21.15.33:0/3914935925 >> v1:172.21.15.167:6789/0 conn(0x7f04c4156c30 legacy=0x7f04c415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 -- 172.21.15.33:0/3914935925 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c406b0d0 con 0x7f04c415acd0 2023-10-06T02:45:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c2d9d700 1 --2- 172.21.15.33:0/3914935925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c4069de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f04c406dbd0 tx=0x7f04bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41426726d7fbce2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c1d9b700 1 -- 172.21.15.33:0/3914935925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00e070 con 0x7f04c415acd0 2023-10-06T02:45:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c1d9b700 1 -- 172.21.15.33:0/3914935925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc00ba20 con 0x7f04c415acd0 2023-10-06T02:45:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c1d9b700 1 -- 172.21.15.33:0/3914935925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0092e0 con 0x7f04c415acd0 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- 172.21.15.33:0/3914935925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 msgr2=0x7f04c4069de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 --2- 172.21.15.33:0/3914935925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c4069de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f04c406dbd0 tx=0x7f04bc009840 comp rx=0 tx=0).stop 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- 172.21.15.33:0/3914935925 shutdown_connections 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 --2- 172.21.15.33:0/3914935925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c4069de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 --2- 172.21.15.33:0/3914935925 >> v2:172.21.15.167:3300/0 conn(0x7f04c4156860 0x7f04c40c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- 172.21.15.33:0/3914935925 >> 172.21.15.33:0/3914935925 conn(0x7f04c40c8ad0 msgr2=0x7f04c40c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- 172.21.15.33:0/3914935925 shutdown_connections 2023-10-06T02:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.445+0000 7f04c359e700 1 -- 172.21.15.33:0/3914935925 wait complete. 2023-10-06T02:45:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 Processor -- start 2023-10-06T02:45:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 -- start start 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c4156860 0x7f04c4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c4156c30 0x7f04c40c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c40b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40c6850 con 0x7f04c415acd0 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41598d0 con 0x7f04c4156c30 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c4159030 con 0x7f04c4156860 2023-10-06T02:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c40b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c40b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55120/0 (socket says 172.21.15.33:55120) 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 -- 172.21.15.33:0/4113014394 learned_addr learned my addr 172.21.15.33:0/4113014394 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04b9d9b700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c4156c30 0x7f04c40c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 -- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c4156860 msgr2=0x7f04c4158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c4156860 0x7f04c4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 -- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c4156c30 msgr2=0x7f04c40c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c4156c30 0x7f04c40c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 -- 172.21.15.33:0/4113014394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc009ae0 con 0x7f04c415acd0 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04b9d9b700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c4156c30 0x7f04c40c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:50.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c2d9d700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c40b1590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009880 tx=0x7f04bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc008be0 con 0x7f04c415acd0 2023-10-06T02:45:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc008d40 con 0x7f04c415acd0 2023-10-06T02:45:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c4159310 con 0x7f04c415acd0 2023-10-06T02:45:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0256b0 con 0x7f04c415acd0 2023-10-06T02:45:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.449+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c40af5a0 con 0x7f04c415acd0 2023-10-06T02:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.453+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c40af790 con 0x7f04c415acd0 2023-10-06T02:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.453+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f04bc003040 con 0x7f04c415acd0 2023-10-06T02:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.453+0000 7f04baffd700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f04a4060550 0x7f04a4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.453+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f04bc07e2d0 con 0x7f04c415acd0 2023-10-06T02:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.453+0000 7f04c259c700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f04a4060550 0x7f04a4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.453+0000 7f04c259c700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f04a4060550 0x7f04a4062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f04c415a000 tx=0x7f04b40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.461+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f04bc04e480 con 0x7f04c415acd0 2023-10-06T02:45:50.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.621+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04c4071040 con 0x7f04c415acd0 2023-10-06T02:45:50.626 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.621+0000 7f04baffd700 1 -- 172.21.15.33:0/4113014394 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04bc051aa0 con 0x7f04c415acd0 2023-10-06T02:45:50.627 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:45:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f04a4060550 msgr2=0x7f04a4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f04a4060550 0x7f04a4062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f04c415a000 tx=0x7f04b40091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 msgr2=0x7f04c40b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c40b1590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009880 tx=0x7f04bc018000 comp rx=0 tx=0).stop 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 shutdown_connections 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04c415acd0 0x7f04c40b1590 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c4156c30 0x7f04c40c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f04a4060550 0x7f04a4062a00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 --2- 172.21.15.33:0/4113014394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c4156860 0x7f04c4158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 >> 172.21.15.33:0/4113014394 conn(0x7f04c40c8ad0 msgr2=0x7f04c4150040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 shutdown_connections 2023-10-06T02:45:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.625+0000 7f04c359e700 1 -- 172.21.15.33:0/4113014394 wait complete. 2023-10-06T02:45:50.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:45:50.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 Processor -- start 2023-10-06T02:45:50.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 -- start start 2023-10-06T02:45:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f431c0cc000 0x7f431c06c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f431c0c7130 0x7f431c0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c1591f0 con 0x7f431c0cc000 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c0c7ad0 con 0x7f431c0c7130 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43226fd700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c068ed0 con 0x7f431c0cc3d0 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43216fb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f431c0cc000 0x7f431c06c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43216fb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f431c0cc000 0x7f431c06c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55128/0 (socket says 172.21.15.33:55128) 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f43216fb700 1 -- 172.21.15.33:0/3734549782 learned_addr learned my addr 172.21.15.33:0/3734549782 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f4320efa700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.745+0000 7f4321efc700 1 --2- 172.21.15.33:0/3734549782 >> v2:172.21.15.167:3300/0 conn(0x7f431c0c7130 0x7f431c0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43216fb700 1 -- 172.21.15.33:0/3734549782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 msgr2=0x7f431c067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43216fb700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c067b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43216fb700 1 -- 172.21.15.33:0/3734549782 >> v2:172.21.15.167:3300/0 conn(0x7f431c0c7130 msgr2=0x7f431c0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43216fb700 1 --2- 172.21.15.33:0/3734549782 >> v2:172.21.15.167:3300/0 conn(0x7f431c0c7130 0x7f431c0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43216fb700 1 -- 172.21.15.33:0/3734549782 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c068070 con 0x7f431c0cc000 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f4320efa700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43216fb700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0cc000 0x7f431c06c3d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f431c0c6eb0 tx=0x7f4318009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1efb752fa44e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f4313fff700 1 -- 172.21.15.33:0/3734549782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318005c10 con 0x7f431c0cc000 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f4313fff700 1 -- 172.21.15.33:0/3734549782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318005d70 con 0x7f431c0cc000 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f4313fff700 1 -- 172.21.15.33:0/3734549782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318006260 con 0x7f431c0cc000 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 -- 172.21.15.33:0/3734549782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0cc000 msgr2=0x7f431c06c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0cc000 0x7f431c06c3d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f431c0c6eb0 tx=0x7f4318009220 comp rx=0 tx=0).stop 2023-10-06T02:45:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 -- 172.21.15.33:0/3734549782 shutdown_connections 2023-10-06T02:45:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 --2- 172.21.15.33:0/3734549782 >> v2:172.21.15.167:3300/0 conn(0x7f431c0c7130 0x7f431c0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 --2- 172.21.15.33:0/3734549782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0cc000 0x7f431c06c3d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 -- 172.21.15.33:0/3734549782 >> 172.21.15.33:0/3734549782 conn(0x7f431c0c9350 msgr2=0x7f431c0c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 -- 172.21.15.33:0/3734549782 shutdown_connections 2023-10-06T02:45:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.749+0000 7f43226fd700 1 -- 172.21.15.33:0/3734549782 wait complete. 2023-10-06T02:45:50.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 Processor -- start 2023-10-06T02:45:50.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 -- start start 2023-10-06T02:45:50.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 0x7f431c157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c0cc000 0x7f431c15bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c168b60 con 0x7f431c0c7130 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c157580 con 0x7f431c0cc000 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c168d70 con 0x7f431c0cc3d0 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 0x7f431c157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 0x7f431c157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55134/0 (socket says 172.21.15.33:55134) 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 -- 172.21.15.33:0/2655703961 learned_addr learned my addr 172.21.15.33:0/2655703961 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f4320efa700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c0cc000 0x7f431c15bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f4321efc700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 -- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 msgr2=0x7f431c0ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c0ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 -- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c0cc000 msgr2=0x7f431c15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c0cc000 0x7f431c15bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 -- 172.21.15.33:0/2655703961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c009420 con 0x7f431c0c7130 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f4320efa700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c0cc000 0x7f431c15bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43216fb700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 0x7f431c157040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4318007fd0 tx=0x7f4318016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318026070 con 0x7f431c0c7130 2023-10-06T02:45:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43180068f0 con 0x7f431c0c7130 2023-10-06T02:45:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180118a0 con 0x7f431c0c7130 2023-10-06T02:45:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4318009480 con 0x7f431c0c7130 2023-10-06T02:45:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.753+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c169330 con 0x7f431c0c7130 2023-10-06T02:45:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.757+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c071040 con 0x7f431c0c7130 2023-10-06T02:45:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.757+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4318011cd0 con 0x7f431c0c7130 2023-10-06T02:45:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.757+0000 7f43117fa700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4300060550 0x7f4300062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.757+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4318086340 con 0x7f431c0c7130 2023-10-06T02:45:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.757+0000 7f4320efa700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4300060550 0x7f4300062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.757+0000 7f4320efa700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4300060550 0x7f4300062a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f430c0093f0 tx=0x7f430c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.769+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f43180564e0 con 0x7f431c0c7130 2023-10-06T02:45:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.925+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f431c06ccd0 con 0x7f431c0c7130 2023-10-06T02:45:50.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43117fa700 1 -- 172.21.15.33:0/2655703961 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f431802f020 con 0x7f431c0c7130 2023-10-06T02:45:50.933 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:45:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4300060550 msgr2=0x7f4300062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4300060550 0x7f4300062a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f430c0093f0 tx=0x7f430c008040 comp rx=0 tx=0).stop 2023-10-06T02:45:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 msgr2=0x7f431c157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 0x7f431c157040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4318007fd0 tx=0x7f4318016000 comp rx=0 tx=0).stop 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 shutdown_connections 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4300060550 0x7f4300062a00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f431c0cc3d0 0x7f431c0ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c0cc000 0x7f431c15bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 --2- 172.21.15.33:0/2655703961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f431c0c7130 0x7f431c157040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 >> 172.21.15.33:0/2655703961 conn(0x7f431c0c9350 msgr2=0x7f431c0c1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 shutdown_connections 2023-10-06T02:45:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:50.929+0000 7f43226fd700 1 -- 172.21.15.33:0/2655703961 wait complete. 2023-10-06T02:45:50.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:45:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.053+0000 7fc8c2a25700 1 Processor -- start 2023-10-06T02:45:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.053+0000 7fc8c2a25700 1 -- start start 2023-10-06T02:45:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.053+0000 7fc8c2a25700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc8bc158c60 0x7fc8bc159030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.053+0000 7fc8c2a25700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc0cd7f0 con 0x7fc8bc15cb70 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.053+0000 7fc8c2a25700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc0cddb0 con 0x7fc8bc0cb1f0 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.053+0000 7fc8c2a25700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8bc15d630 con 0x7fc8bc158c60 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc8bc15cb70 0x7fc8bc0caae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44880/0 (socket says 172.21.15.33:44880) 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 -- 172.21.15.33:0/2543793652 learned_addr learned my addr 172.21.15.33:0/2543793652 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1a23700 1 --2- 172.21.15.33:0/2543793652 >> v2:172.21.15.167:3300/0 conn(0x7fc8bc158c60 0x7fc8bc159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155048316 0 0) 0x7fc8bc0cd7f0 con 0x7fc8bc15cb70 2023-10-06T02:45:51.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a8003570 con 0x7fc8bc15cb70 2023-10-06T02:45:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565675097 0 0) 0x7fc8a8003570 con 0x7fc8bc15cb70 2023-10-06T02:45:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 >> v2:172.21.15.167:3300/0 conn(0x7fc8bc158c60 msgr2=0x7fc8bc159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 --2- 172.21.15.33:0/2543793652 >> v2:172.21.15.167:3300/0 conn(0x7fc8bc158c60 0x7fc8bc159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 >> v1:172.21.15.167:6789/0 conn(0x7fc8bc0cb1f0 legacy=0x7fc8bc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc15d910 con 0x7fc8bc15cb70 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7fc8b0001470 con 0x7fc8bc15cb70 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 >> v1:172.21.15.33:6789/0 conn(0x7fc8bc15cb70 legacy=0x7fc8bc0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8003a60 0x7fc8a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8a8004810 0x7fc8a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a80051c0 0x7fc8a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc15d630 con 0x7fc8a8004810 2023-10-06T02:45:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a800c050 con 0x7fc8a8003a60 2023-10-06T02:45:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a800c210 con 0x7fc8a80051c0 2023-10-06T02:45:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8a8004810 0x7fc8a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1a23700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8003a60 0x7fc8a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2224700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a80051c0 0x7fc8a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 -- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a80051c0 msgr2=0x7fc8a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a80051c0 0x7fc8a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 -- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8003a60 msgr2=0x7fc8a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8003a60 0x7fc8a8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 -- 172.21.15.33:0/2543793652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a8003570 con 0x7fc8a8004810 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2224700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a80051c0 0x7fc8a800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1a23700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8003a60 0x7fc8a8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2224700 1 -- 172.21.15.33:0/2543793652 reap_dead start 2023-10-06T02:45:51.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c1222700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8a8004810 0x7fc8a8004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c3c0 tx=0x7fc8b0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf447837535446d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000cc90 con 0x7fc8a8004810 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b000cdf0 con 0x7fc8a8004810 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c0a21700 1 -- 172.21.15.33:0/2543793652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0005b70 con 0x7fc8a8004810 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 -- 172.21.15.33:0/2543793652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8a8004810 msgr2=0x7fc8a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8a8004810 0x7fc8a8004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c3c0 tx=0x7fc8b0001e70 comp rx=0 tx=0).stop 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 -- 172.21.15.33:0/2543793652 shutdown_connections 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/2543793652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8a8004810 0x7fc8a8004c80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 -- 172.21.15.33:0/2543793652 >> 172.21.15.33:0/2543793652 conn(0x7fc8bc067a80 msgr2=0x7fc8bc0cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 -- 172.21.15.33:0/2543793652 shutdown_connections 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.057+0000 7fc8c2a25700 1 -- 172.21.15.33:0/2543793652 wait complete. 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 Processor -- start 2023-10-06T02:45:51.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 -- start start 2023-10-06T02:45:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8004810 0x7fc8bc150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 0x7fc8bc157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8bc157c60 0x7fc8bc153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc157290 con 0x7fc8bc15ba70 2023-10-06T02:45:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0b3ca0 con 0x7fc8a8004810 2023-10-06T02:45:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0b3f80 con 0x7fc8bc157c60 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 0x7fc8bc157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 0x7fc8bc157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55150/0 (socket says 172.21.15.33:55150) 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 -- 172.21.15.33:0/4283626320 learned_addr learned my addr 172.21.15.33:0/4283626320 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2224700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8bc157c60 0x7fc8bc153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1a23700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8004810 0x7fc8bc150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 -- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8bc157c60 msgr2=0x7fc8bc153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8bc157c60 0x7fc8bc153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 -- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8004810 msgr2=0x7fc8bc150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8004810 0x7fc8bc150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 -- 172.21.15.33:0/4283626320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc0b4260 con 0x7fc8bc15ba70 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2224700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8bc157c60 0x7fc8bc153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c1222700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 0x7fc8bc157720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c9b0 tx=0x7fc8b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0001430 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b00070d0 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc156f20 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0007490 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.061+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc0b3a10 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.065+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc071040 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.065+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc8b0012400 con 0x7fc8bc15ba70 2023-10-06T02:45:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.065+0000 7fc8adffb700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc8a0060550 0x7fc8a0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.065+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fc8b007d570 con 0x7fc8bc15ba70 2023-10-06T02:45:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.065+0000 7fc8c1a23700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc8a0060550 0x7fc8a0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.069+0000 7fc8c1a23700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc8a0060550 0x7fc8a0062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc065810 tx=0x7fc8b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.073+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc8b004d7c0 con 0x7fc8bc15ba70 2023-10-06T02:45:51.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.233+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc8bc0cda40 con 0x7fc8bc15ba70 2023-10-06T02:45:51.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.233+0000 7fc8adffb700 1 -- 172.21.15.33:0/4283626320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc8b0050de0 con 0x7fc8bc15ba70 2023-10-06T02:45:51.240 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:45:51.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc8a0060550 msgr2=0x7fc8a0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc8a0060550 0x7fc8a0062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc065810 tx=0x7fc8b8008040 comp rx=0 tx=0).stop 2023-10-06T02:45:51.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 msgr2=0x7fc8bc157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 0x7fc8bc157720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c9b0 tx=0x7fc8b0015000 comp rx=0 tx=0).stop 2023-10-06T02:45:51.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 shutdown_connections 2023-10-06T02:45:51.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc8a0060550 0x7fc8a0062a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8bc157c60 0x7fc8bc153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8bc15ba70 0x7fc8bc157720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 --2- 172.21.15.33:0/4283626320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a8004810 0x7fc8bc150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 >> 172.21.15.33:0/4283626320 conn(0x7fc8bc067a80 msgr2=0x7fc8bc0cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:51.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 shutdown_connections 2023-10-06T02:45:51.246 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.237+0000 7fc8c2a25700 1 -- 172.21.15.33:0/4283626320 wait complete. 2023-10-06T02:45:51.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 Processor -- start 2023-10-06T02:45:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 -- start start 2023-10-06T02:45:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f826806d590 0x7f82680cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82680cc3d0 0x7f82680c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8268068fb0 con 0x7f826806d1c0 2023-10-06T02:45:51.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8268069b50 con 0x7f826806d590 2023-10-06T02:45:51.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8267fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82680680c0 con 0x7f82680cc3d0 2023-10-06T02:45:51.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8266ffd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f826806d1c0 0x7f8268069440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:33812/0 (socket says 172.21.15.33:33812) 2023-10-06T02:45:51.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8266ffd700 1 -- 172.21.15.33:0/925883213 learned_addr learned my addr 172.21.15.33:0/925883213 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:51.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f82667fc700 1 --2- 172.21.15.33:0/925883213 >> v2:172.21.15.33:3300/0 conn(0x7f826806d590 0x7f82680cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f82677fe700 1 --2- 172.21.15.33:0/925883213 >> v2:172.21.15.167:3300/0 conn(0x7f82680cc3d0 0x7f82680c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8265ffb700 1 -- 172.21.15.33:0/925883213 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749841992 0 0) 0x7f8268068fb0 con 0x7f826806d1c0 2023-10-06T02:45:51.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f8265ffb700 1 -- 172.21.15.33:0/925883213 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8254003570 con 0x7f826806d1c0 2023-10-06T02:45:51.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f82667fc700 1 -- 172.21.15.33:0/925883213 >> v2:172.21.15.167:3300/0 conn(0x7f82680cc3d0 msgr2=0x7f82680c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:51.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f82667fc700 1 --2- 172.21.15.33:0/925883213 >> v2:172.21.15.167:3300/0 conn(0x7f82680cc3d0 0x7f82680c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f82667fc700 1 -- 172.21.15.33:0/925883213 >> v1:172.21.15.167:6789/0 conn(0x7f826806d1c0 legacy=0x7f8268069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.353+0000 7f82667fc700 1 -- 172.21.15.33:0/925883213 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680683a0 con 0x7f826806d590 2023-10-06T02:45:51.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f82667fc700 1 --2- 172.21.15.33:0/925883213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d590 0x7f82680cbe90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f82680ceb30 tx=0x7f82500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccbe8de0b1e4cbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8265ffb700 1 -- 172.21.15.33:0/925883213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250011630 con 0x7f826806d590 2023-10-06T02:45:51.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8265ffb700 1 -- 172.21.15.33:0/925883213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500117b0 con 0x7f826806d590 2023-10-06T02:45:51.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8265ffb700 1 -- 172.21.15.33:0/925883213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250011b70 con 0x7f826806d590 2023-10-06T02:45:51.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- 172.21.15.33:0/925883213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d590 msgr2=0x7f82680cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 --2- 172.21.15.33:0/925883213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d590 0x7f82680cbe90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f82680ceb30 tx=0x7f82500091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- 172.21.15.33:0/925883213 shutdown_connections 2023-10-06T02:45:51.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 --2- 172.21.15.33:0/925883213 >> v2:172.21.15.167:3300/0 conn(0x7f82680cc3d0 0x7f82680c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 --2- 172.21.15.33:0/925883213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d590 0x7f82680cbe90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- 172.21.15.33:0/925883213 >> 172.21.15.33:0/925883213 conn(0x7f826806c280 msgr2=0x7f82680cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:51.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- 172.21.15.33:0/925883213 shutdown_connections 2023-10-06T02:45:51.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- 172.21.15.33:0/925883213 wait complete. 2023-10-06T02:45:51.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 Processor -- start 2023-10-06T02:45:51.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- start start 2023-10-06T02:45:51.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 0x7f826815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f826806d590 0x7f82680ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82680cc3d0 0x7f8268156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815e260 con 0x7f826806d1c0 2023-10-06T02:45:51.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815dca0 con 0x7f82680cc3d0 2023-10-06T02:45:51.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8267fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826815df80 con 0x7f826806d590 2023-10-06T02:45:51.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 0x7f826815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 0x7f826815c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55168/0 (socket says 172.21.15.33:55168) 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 -- 172.21.15.33:0/1542392028 learned_addr learned my addr 172.21.15.33:0/1542392028 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f82667fc700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f826806d590 0x7f82680ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f82677fe700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82680cc3d0 0x7f8268156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 -- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f826806d590 msgr2=0x7f82680ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f826806d590 0x7f82680ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 -- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82680cc3d0 msgr2=0x7f8268156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82680cc3d0 0x7f8268156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 -- 172.21.15.33:0/1542392028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009420 con 0x7f826806d1c0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f82677fe700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82680cc3d0 0x7f8268156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f8266ffd700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 0x7f826815c0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f825800cc40 tx=0x7f8258001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800a910 con 0x7f826806d1c0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.357+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825800aa70 con 0x7f826806d1c0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82680c4c20 con 0x7f826806d1c0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800ae30 con 0x7f826806d1c0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826815c610 con 0x7f826806d1c0 2023-10-06T02:45:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8268071040 con 0x7f826806d1c0 2023-10-06T02:45:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8258007300 con 0x7f826806d1c0 2023-10-06T02:45:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f825effd700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8248060550 0x7f8248062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f825807cf60 con 0x7f826806d1c0 2023-10-06T02:45:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f82667fc700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8248060550 0x7f8248062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.361+0000 7f82667fc700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8248060550 0x7f8248062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82680ceb30 tx=0x7f8250017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.365+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f825804d180 con 0x7f826806d1c0 2023-10-06T02:45:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.565+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f826806c680 con 0x7f826806d1c0 2023-10-06T02:45:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.565+0000 7f825effd700 1 -- 172.21.15.33:0/1542392028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f82580507a0 con 0x7f826806d1c0 2023-10-06T02:45:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:51.572 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:45:51.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8248060550 msgr2=0x7f8248062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8248060550 0x7f8248062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82680ceb30 tx=0x7f8250017000 comp rx=0 tx=0).stop 2023-10-06T02:45:51.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 msgr2=0x7f826815c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 0x7f826815c0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f825800cc40 tx=0x7f8258001a70 comp rx=0 tx=0).stop 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 shutdown_connections 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8248060550 0x7f8248062a00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f826806d590 0x7f82680ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82680cc3d0 0x7f8268156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 --2- 172.21.15.33:0/1542392028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f826806d1c0 0x7f826815c0d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 >> 172.21.15.33:0/1542392028 conn(0x7f826806c280 msgr2=0x7f82681573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 shutdown_connections 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.569+0000 7f8267fff700 1 -- 172.21.15.33:0/1542392028 wait complete. 2023-10-06T02:45:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:45:51.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:45:51.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 Processor -- start 2023-10-06T02:45:51.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 -- start start 2023-10-06T02:45:51.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f984c157280 0x7f984c0aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c0679c0 0x7f984c067e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f984c157650 0x7f984c157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c0c6c10 con 0x7f984c157280 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c068350 con 0x7f984c157650 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f98523f3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c157ff0 con 0x7f984c0679c0 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f984c157280 0x7f984c0aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f984c157280 0x7f984c0aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55174/0 (socket says 172.21.15.33:55174) 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 -- 172.21.15.33:0/387204011 learned_addr learned my addr 172.21.15.33:0/387204011 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984b7fe700 1 --2- 172.21.15.33:0/387204011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c0679c0 0x7f984c067e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f9850990700 1 --2- 172.21.15.33:0/387204011 >> v2:172.21.15.167:3300/0 conn(0x7f984c157650 0x7f984c157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 -- 172.21.15.33:0/387204011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c0679c0 msgr2=0x7f984c067e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:51.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 --2- 172.21.15.33:0/387204011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c0679c0 0x7f984c067e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 -- 172.21.15.33:0/387204011 >> v2:172.21.15.167:3300/0 conn(0x7f984c157650 msgr2=0x7f984c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:51.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 --2- 172.21.15.33:0/387204011 >> v2:172.21.15.167:3300/0 conn(0x7f984c157650 0x7f984c157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 -- 172.21.15.33:0/387204011 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c0685d0 con 0x7f984c157280 2023-10-06T02:45:51.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984bfff700 1 --2- 172.21.15.33:0/387204011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c157280 0x7f984c0aadd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c7ae0 tx=0x7f9844009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb226fe6b73a5aeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984affd700 1 -- 172.21.15.33:0/387204011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400e070 con 0x7f984c157280 2023-10-06T02:45:51.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984affd700 1 -- 172.21.15.33:0/387204011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984400b970 con 0x7f984c157280 2023-10-06T02:45:51.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.693+0000 7f984affd700 1 -- 172.21.15.33:0/387204011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98440092e0 con 0x7f984c157280 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 -- 172.21.15.33:0/387204011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c157280 msgr2=0x7f984c0aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 --2- 172.21.15.33:0/387204011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c157280 0x7f984c0aadd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c7ae0 tx=0x7f9844009840 comp rx=0 tx=0).stop 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 -- 172.21.15.33:0/387204011 shutdown_connections 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 --2- 172.21.15.33:0/387204011 >> v2:172.21.15.167:3300/0 conn(0x7f984c157650 0x7f984c157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 --2- 172.21.15.33:0/387204011 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c0679c0 0x7f984c067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 --2- 172.21.15.33:0/387204011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c157280 0x7f984c0aadd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 -- 172.21.15.33:0/387204011 >> 172.21.15.33:0/387204011 conn(0x7f984c0c83f0 msgr2=0x7f984c0c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 -- 172.21.15.33:0/387204011 shutdown_connections 2023-10-06T02:45:51.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 -- 172.21.15.33:0/387204011 wait complete. 2023-10-06T02:45:51.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.697+0000 7f98523f3700 1 Processor -- start 2023-10-06T02:45:51.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 -- start start 2023-10-06T02:45:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 0x7f984c158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f984c157280 0x7f984c0c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c157650 0x7f984c146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0c3a00 con 0x7f984c0679c0 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0c4860 con 0x7f984c157280 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0c3c50 con 0x7f984c157650 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 0x7f984c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 0x7f984c158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55186/0 (socket says 172.21.15.33:55186) 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 -- 172.21.15.33:0/1998901346 learned_addr learned my addr 172.21.15.33:0/1998901346 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984b7fe700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f984c157280 0x7f984c0c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f9850990700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c157650 0x7f984c146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 -- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c157650 msgr2=0x7f984c146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c157650 0x7f984c146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 -- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f984c157280 msgr2=0x7f984c0c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f984c157280 0x7f984c0c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 -- 172.21.15.33:0/1998901346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009ae0 con 0x7f984c0679c0 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f984bfff700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 0x7f984c158320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c29e0 tx=0x7f9844018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844008950 con 0x7f984c0679c0 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844008ab0 con 0x7f984c0679c0 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c0c3f30 con 0x7f984c0679c0 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844008e70 con 0x7f984c0679c0 2023-10-06T02:45:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.701+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c0c44f0 con 0x7f984c0679c0 2023-10-06T02:45:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.705+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c0710b0 con 0x7f984c0679c0 2023-10-06T02:45:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.705+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9844003040 con 0x7f984c0679c0 2023-10-06T02:45:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.705+0000 7f9843fff700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9830060550 0x7f9830062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.705+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f984407e9f0 con 0x7f984c0679c0 2023-10-06T02:45:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.705+0000 7f984b7fe700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9830060550 0x7f9830062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.705+0000 7f984b7fe700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9830060550 0x7f9830062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c7c20 tx=0x7f983c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.709+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f984404eaf0 con 0x7f984c0679c0 2023-10-06T02:45:51.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.873+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f984c0688b0 con 0x7f984c0679c0 2023-10-06T02:45:51.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.873+0000 7f9843fff700 1 -- 172.21.15.33:0/1998901346 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18707 (secure 0 0 0) 0x7f9844052110 con 0x7f984c0679c0 2023-10-06T02:45:51.879 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:51.879 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:45:27.680466+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9830060550 msgr2=0x7f9830062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9830060550 0x7f9830062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c7c20 tx=0x7f983c008040 comp rx=0 tx=0).stop 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 msgr2=0x7f984c158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 0x7f984c158320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c29e0 tx=0x7f9844018000 comp rx=0 tx=0).stop 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 shutdown_connections 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9830060550 0x7f9830062a00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f984c157650 0x7f984c146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f984c157280 0x7f984c0c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 --2- 172.21.15.33:0/1998901346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c0679c0 0x7f984c158320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 >> 172.21.15.33:0/1998901346 conn(0x7f984c0c83f0 msgr2=0x7f984c156270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 shutdown_connections 2023-10-06T02:45:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.877+0000 7f98523f3700 1 -- 172.21.15.33:0/1998901346 wait complete. 2023-10-06T02:45:51.892 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:45:51.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 Processor -- start 2023-10-06T02:45:51.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 -- start start 2023-10-06T02:45:52.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa8a806b050 0x7fa8a8066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a80c67e0 0x7fa8a80c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a806b420 con 0x7fa8a80c93a0 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a80674f0 con 0x7fa8a806b050 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8aee35700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a806a4e0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ad632700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa8a80c93a0 0x7fa8a80c60d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44882/0 (socket says 172.21.15.33:44882) 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ad632700 1 -- 172.21.15.33:0/3096488993 learned_addr learned my addr 172.21.15.33:0/3096488993 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ade33700 1 --2- 172.21.15.33:0/3096488993 >> v2:172.21.15.33:3300/0 conn(0x7fa8a806b050 0x7fa8a8066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ae634700 1 --2- 172.21.15.33:0/3096488993 >> v2:172.21.15.167:3300/0 conn(0x7fa8a80c67e0 0x7fa8a80c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ace31700 1 -- 172.21.15.33:0/3096488993 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295437966 0 0) 0x7fa8a806b420 con 0x7fa8a80c93a0 2023-10-06T02:45:52.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ace31700 1 -- 172.21.15.33:0/3096488993 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa894003570 con 0x7fa8a80c93a0 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ade33700 1 -- 172.21.15.33:0/3096488993 >> v2:172.21.15.167:3300/0 conn(0x7fa8a80c67e0 msgr2=0x7fa8a80c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ade33700 1 --2- 172.21.15.33:0/3096488993 >> v2:172.21.15.167:3300/0 conn(0x7fa8a80c67e0 0x7fa8a80c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ade33700 1 -- 172.21.15.33:0/3096488993 >> v1:172.21.15.33:6789/0 conn(0x7fa8a80c93a0 legacy=0x7fa8a80c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ade33700 1 -- 172.21.15.33:0/3096488993 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a8067700 con 0x7fa8a806b050 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ade33700 1 --2- 172.21.15.33:0/3096488993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a806b050 0x7fa8a8066fb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80cb020 tx=0x7fa89c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2fe1ef70774a82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ace31700 1 -- 172.21.15.33:0/3096488993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011630 con 0x7fa8a806b050 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ace31700 1 -- 172.21.15.33:0/3096488993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c0117b0 con 0x7fa8a806b050 2023-10-06T02:45:52.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.993+0000 7fa8ace31700 1 -- 172.21.15.33:0/3096488993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011b30 con 0x7fa8a806b050 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 -- 172.21.15.33:0/3096488993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a806b050 msgr2=0x7fa8a8066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 --2- 172.21.15.33:0/3096488993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a806b050 0x7fa8a8066fb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80cb020 tx=0x7fa89c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 -- 172.21.15.33:0/3096488993 shutdown_connections 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 --2- 172.21.15.33:0/3096488993 >> v2:172.21.15.167:3300/0 conn(0x7fa8a80c67e0 0x7fa8a80c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 --2- 172.21.15.33:0/3096488993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a806b050 0x7fa8a8066fb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 -- 172.21.15.33:0/3096488993 >> 172.21.15.33:0/3096488993 conn(0x7fa8a80c8240 msgr2=0x7fa8a80c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 -- 172.21.15.33:0/3096488993 shutdown_connections 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 -- 172.21.15.33:0/3096488993 wait complete. 2023-10-06T02:45:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 Processor -- start 2023-10-06T02:45:52.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:51.997+0000 7fa8aee35700 1 -- start start 2023-10-06T02:45:52.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a806b050 0x7fa8a80af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 0x7fa8a80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a80c93a0 0x7fa8a80b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a816cae0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a816d900 con 0x7fa8a80c93a0 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a816ccf0 con 0x7fa8a806b050 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 0x7fa8a80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 0x7fa8a80b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55204/0 (socket says 172.21.15.33:55204) 2023-10-06T02:45:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 -- 172.21.15.33:0/4117757911 learned_addr learned my addr 172.21.15.33:0/4117757911 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ade33700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a806b050 0x7fa8a80af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ae634700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a80c93a0 0x7fa8a80b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 -- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a806b050 msgr2=0x7fa8a80af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a806b050 0x7fa8a80af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 -- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a80c93a0 msgr2=0x7fa8a80b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a80c93a0 0x7fa8a80b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 -- 172.21.15.33:0/4117757911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c009420 con 0x7fa8a80c67e0 2023-10-06T02:45:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8ad632700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 0x7fa8a80b0bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40011e0 tx=0x7fa8a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4005b30 con 0x7fa8a80c67e0 2023-10-06T02:45:52.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a4005c90 con 0x7fa8a80c67e0 2023-10-06T02:45:52.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a816cfd0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4007300 con 0x7fa8a80c67e0 2023-10-06T02:45:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.001+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a816d590 con 0x7fa8a80c67e0 2023-10-06T02:45:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.005+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a8071040 con 0x7fa8a80c67e0 2023-10-06T02:45:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.005+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa8a4007670 con 0x7fa8a80c67e0 2023-10-06T02:45:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.005+0000 7fa899ffb700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.005+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa8a407c0b0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.005+0000 7fa8ade33700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.005+0000 7fa8ade33700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80cb020 tx=0x7fa89c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.013+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa8a404c1b0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.209+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8a80679e0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa899ffb700 1 -- 172.21.15.33:0/4117757911 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fa8a404f7d0 con 0x7fa8a80c67e0 2023-10-06T02:45:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:45:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:45:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 msgr2=0x7fa88c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80cb020 tx=0x7fa89c017000 comp rx=0 tx=0).stop 2023-10-06T02:45:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 msgr2=0x7fa8a80b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 0x7fa8a80b0bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40011e0 tx=0x7fa8a4007df0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 shutdown_connections 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a80c93a0 0x7fa8a80b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80c67e0 0x7fa8a80b0bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 --2- 172.21.15.33:0/4117757911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a806b050 0x7fa8a80af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 >> 172.21.15.33:0/4117757911 conn(0x7fa8a80c8240 msgr2=0x7fa8a815c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 shutdown_connections 2023-10-06T02:45:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.213+0000 7fa8aee35700 1 -- 172.21.15.33:0/4117757911 wait complete. 2023-10-06T02:45:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:45:52.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:45:23.146455+0000', 'modified': '2023-10-06T02:45:41.377151+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/2898450531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 2898450531}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 2898450531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:45:52.634 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:45:52.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T02:45:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.749+0000 7fb902a4c700 1 Processor -- start 2023-10-06T02:45:52.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.749+0000 7fb902a4c700 1 -- start start 2023-10-06T02:45:52.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8fc069580 0x7fb8fc159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc156fe0 con 0x7fb8fc155d30 2023-10-06T02:45:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc06a970 con 0x7fb8fc069580 2023-10-06T02:45:52.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc06a380 con 0x7fb8fc156100 2023-10-06T02:45:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8fb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8fc069580 0x7fb8fc159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8fb7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0687d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:58952/0 (socket says 172.21.15.33:58952) 2023-10-06T02:45:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8fbfff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb8fc155d30 0x7fb8fc0680c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:46146/0 (socket says 172.21.15.33:46146) 2023-10-06T02:45:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8fbfff700 1 -- 172.21.15.33:0/2315237042 learned_addr learned my addr 172.21.15.33:0/2315237042 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:52.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8fc069580 0x7fb8fc159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44542/0 (socket says 172.21.15.33:44542) 2023-10-06T02:45:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 -- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 msgr2=0x7fb8fc0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 --2- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 -- 172.21.15.33:0/2315237042 >> v1:172.21.15.167:6789/0 conn(0x7fb8fc155d30 legacy=0x7fb8fc0680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 -- 172.21.15.33:0/2315237042 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc1572c0 con 0x7fb8fc069580 2023-10-06T02:45:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:45:52.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb900fe9700 1 --2- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc069580 0x7fb8fc159aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00bb90 tx=0x7fb8ec00b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ab75b120cd55e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8faffd700 1 -- 172.21.15.33:0/2315237042 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00db40 con 0x7fb8fc069580 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8faffd700 1 -- 172.21.15.33:0/2315237042 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec00dca0 con 0x7fb8fc069580 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb8faffd700 1 -- 172.21.15.33:0/2315237042 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0196b0 con 0x7fb8fc069580 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc069580 msgr2=0x7fb8fc159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 --2- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc069580 0x7fb8fc159aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00bb90 tx=0x7fb8ec00b430 comp rx=0 tx=0).stop 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- 172.21.15.33:0/2315237042 shutdown_connections 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 --2- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc069580 0x7fb8fc159aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 --2- 172.21.15.33:0/2315237042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- 172.21.15.33:0/2315237042 >> 172.21.15.33:0/2315237042 conn(0x7fb8fc0af920 msgr2=0x7fb8fc0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- 172.21.15.33:0/2315237042 shutdown_connections 2023-10-06T02:45:52.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.753+0000 7fb902a4c700 1 -- 172.21.15.33:0/2315237042 wait complete. 2023-10-06T02:45:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 Processor -- start 2023-10-06T02:45:52.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 -- start start 2023-10-06T02:45:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 0x7fb8fc149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc155d30 0x7fb8fc0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc097770 con 0x7fb8fc069580 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc14a510 con 0x7fb8fc155d30 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb902a4c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc0aa200 con 0x7fb8fc156100 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 0x7fb8fc149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 0x7fb8fc149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35264/0 (socket says 172.21.15.33:35264) 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb8fbfff700 1 -- 172.21.15.33:0/1217784371 learned_addr learned my addr 172.21.15.33:0/1217784371 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc155d30 0x7fb8fc0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.757+0000 7fb900fe9700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8fbfff700 1 -- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 msgr2=0x7fb8fc0a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8fbfff700 1 -- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc155d30 msgr2=0x7fb8fc0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc155d30 0x7fb8fc0a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8fbfff700 1 -- 172.21.15.33:0/1217784371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4009a20 con 0x7fb8fc069580 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 0x7fb8fc149fd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000c00 tx=0x7fb8e4007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8f3fff700 1 -- 172.21.15.33:0/1217784371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4005b80 con 0x7fb8fc069580 2023-10-06T02:45:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8f3fff700 1 -- 172.21.15.33:0/1217784371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e4005ce0 con 0x7fb8fc069580 2023-10-06T02:45:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec00b6d0 con 0x7fb8fc069580 2023-10-06T02:45:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb8f3fff700 1 -- 172.21.15.33:0/1217784371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4007320 con 0x7fb8fc069580 2023-10-06T02:45:52.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.761+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc0aa7c0 con 0x7fb8fc069580 2023-10-06T02:45:52.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.765+0000 7fb8f3fff700 1 -- 172.21.15.33:0/1217784371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb8e4007690 con 0x7fb8fc069580 2023-10-06T02:45:52.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.765+0000 7fb8f3fff700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb8e0060550 0x7fb8e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.765+0000 7fb8f3fff700 1 -- 172.21.15.33:0/1217784371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb8e407bfe0 con 0x7fb8fc069580 2023-10-06T02:45:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.765+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb8e0060550 0x7fb8e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.765+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb8e0060550 0x7fb8e0062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009fd0 tx=0x7fb8f400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.765+0000 7fb904cf5740 1 Processor -- start 2023-10-06T02:45:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- start start 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x2314450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x2314990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x2314ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x223df30 con 0x20dd790 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x2239ed0 con 0x2092710 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x22000b0 con 0x2092ae0 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x2314ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb8fb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x2314990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x2314ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35278/0 (socket says 172.21.15.33:35278) 2023-10-06T02:45:52.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x2314450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 -- 172.21.15.33:0/2965158159 learned_addr learned my addr 172.21.15.33:0/2965158159 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 -- 172.21.15.33:0/2965158159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 msgr2=0x2314450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x2314450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 -- 172.21.15.33:0/2965158159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 msgr2=0x2314990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x2314990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 -- 172.21.15.33:0/2965158159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20de9e0 con 0x20dd790 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb900fe9700 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x2314ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00b6a0 tx=0x7fb8ec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa5bf4596ca8ec3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb8f17fa700 1 -- 172.21.15.33:0/2965158159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00de10 con 0x20dd790 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb8f17fa700 1 -- 172.21.15.33:0/2965158159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec01e050 con 0x20dd790 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb8f17fa700 1 -- 172.21.15.33:0/2965158159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec019a20 con 0x20dd790 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- 172.21.15.33:0/2965158159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 msgr2=0x2314ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x2314ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00b6a0 tx=0x7fb8ec01b000 comp rx=0 tx=0).stop 2023-10-06T02:45:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- 172.21.15.33:0/2965158159 shutdown_connections 2023-10-06T02:45:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x2314ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x2314990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 --2- 172.21.15.33:0/2965158159 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x2314450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- 172.21.15.33:0/2965158159 >> 172.21.15.33:0/2965158159 conn(0x2092eb0 msgr2=0x22cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- 172.21.15.33:0/2965158159 shutdown_connections 2023-10-06T02:45:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.769+0000 7fb904cf5740 1 -- 172.21.15.33:0/2965158159 wait complete. 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 -- start start 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 Processor -- start 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x22f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22f5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x22f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x21ffd50 con 0x20dd790 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x21d8970 con 0x2092710 2023-10-06T02:45:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x21f7d70 con 0x2092ae0 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb904cf5740 10 client.0 _ll_register_callbacks cb 0x2283b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x22f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x22f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35286/0 (socket says 172.21.15.33:35286) 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 -- 172.21.15.33:0/1624242244 learned_addr learned my addr 172.21.15.33:0/1624242244 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x22f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 -- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 msgr2=0x22f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 -- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 msgr2=0x22f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x22f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 -- 172.21.15.33:0/1624242244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc1572c0 con 0x20dd790 2023-10-06T02:45:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb900fe9700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x22f56b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00b6a0 tx=0x7fb8ec022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39661367d374296 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.773+0000 7fb8ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T02:45:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1624242244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00db40 con 0x20dd790 2023-10-06T02:45:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1624242244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb8ec014070 con 0x20dd790 2023-10-06T02:45:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21d8cc0 con 0x20dd790 2023-10-06T02:45:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1624242244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8ec024690 con 0x20dd790 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 10 client.4806 fetch_fsmap learned FSMap version 9 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x2208370 con 0x20dd790 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1624242244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21d8cc0 con 0x20dd790 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1624242244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 9) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fb8ec0247f0 con 0x20dd790 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 10 client.4806 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 20 client.4806 populate_metadata read hostname 'smithi033' 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x2320cb0 0x2323090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 4 client.4806 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 --> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21dc3e0 con 0x2320cb0 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x2320cb0 0x2323090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 10 client.4806 ms_handle_connect on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x2320cb0 0x2323090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a901301075fb03f server_cookie=92a3e4d7417ccec1 in_seq=0 out_seq=0 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1624242244 <== mds.0 v2:172.21.15.33:6834/2898450531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x21dc3e0 con 0x2320cb0 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 10 client.4806 handle_command_reply: tid=0 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 client.4806 shutdown 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 msgr2=0x22f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x22f56b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00b6a0 tx=0x7fb8ec022000 comp rx=0 tx=0).stop 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 shutdown_connections 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x2320cb0 0x2323090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x20dd790 0x22f56b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1624242244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x2092710 0x22f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb8ea7fc700 0 client.4806 ms_handle_reset on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 >> 172.21.15.33:0/1624242244 conn(0x2092eb0 msgr2=0x22c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 shutdown_connections 2023-10-06T02:45:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 1 -- 172.21.15.33:0/1624242244 wait complete. 2023-10-06T02:45:52.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.777+0000 7fb904cf5740 20 client.4806 trim_cache size 0 max 16384 2023-10-06T02:45:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 Processor -- start 2023-10-06T02:45:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- start start 2023-10-06T02:45:52.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 0x22cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x220df40 con 0x2092710 2023-10-06T02:45:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x223e490 con 0x20dd790 2023-10-06T02:45:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x2235b50 con 0x2092ae0 2023-10-06T02:45:52.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 0x22cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 0x22cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35296/0 (socket says 172.21.15.33:35296) 2023-10-06T02:45:52.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 -- 172.21.15.33:0/3302592711 learned_addr learned my addr 172.21.15.33:0/3302592711 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:52.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb900fe9700 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 -- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 msgr2=0x22cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 -- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 msgr2=0x22eda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22eda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 -- 172.21.15.33:0/3302592711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc0b0050 con 0x2092710 2023-10-06T02:45:52.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 0x22cb960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb8e407a0c0 tx=0x7fb8e4063040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b23b4607dd5acb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8f1ffb700 1 -- 172.21.15.33:0/3302592711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4053050 con 0x2092710 2023-10-06T02:45:52.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8f1ffb700 1 -- 172.21.15.33:0/3302592711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e407a140 con 0x2092710 2023-10-06T02:45:52.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb8f1ffb700 1 -- 172.21.15.33:0/3302592711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e404fdc0 con 0x2092710 2023-10-06T02:45:52.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- 172.21.15.33:0/3302592711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 msgr2=0x22cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 0x22cb960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb8e407a0c0 tx=0x7fb8e4063040 comp rx=0 tx=0).stop 2023-10-06T02:45:52.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- 172.21.15.33:0/3302592711 shutdown_connections 2023-10-06T02:45:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092ae0 0x22cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 --2- 172.21.15.33:0/3302592711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092710 0x22cb960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.797+0000 7fb904cf5740 1 -- 172.21.15.33:0/3302592711 >> 172.21.15.33:0/3302592711 conn(0x2092eb0 msgr2=0x22ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 -- 172.21.15.33:0/3302592711 shutdown_connections 2023-10-06T02:45:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 -- 172.21.15.33:0/3302592711 wait complete. 2023-10-06T02:45:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 -- start start 2023-10-06T02:45:52.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 Processor -- start 2023-10-06T02:45:52.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092710 0x22efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 0x22f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x220e610 con 0x2092ae0 2023-10-06T02:45:52.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x210b7c0 con 0x20dd790 2023-10-06T02:45:52.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x22a7bb0 con 0x2092710 2023-10-06T02:45:52.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb904cf5740 10 client.0 _ll_register_callbacks cb 0x21560e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:45:52.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb8fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 0x22f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb8fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 0x22f0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35312/0 (socket says 172.21.15.33:35312) 2023-10-06T02:45:52.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb8fb7fe700 1 -- 172.21.15.33:0/1167865445 learned_addr learned my addr 172.21.15.33:0/1167865445 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:52.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb900fe9700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092710 0x22efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb8fb7fe700 1 -- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092710 msgr2=0x22efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.801+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092710 0x22efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8fb7fe700 1 -- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 msgr2=0x22f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8fb7fe700 1 -- 172.21.15.33:0/1167865445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc0aa7c0 con 0x2092ae0 2023-10-06T02:45:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8fb7fe700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 0x22f0400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4015fa0 tx=0x7fb8f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647f1cc098768079 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T02:45:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1167865445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400e040 con 0x2092ae0 2023-10-06T02:45:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8ec01e050 con 0x2092ae0 2023-10-06T02:45:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1167865445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fb8f4026da0 con 0x2092ae0 2023-10-06T02:45:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1167865445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8f4017650 con 0x2092ae0 2023-10-06T02:45:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 10 client.4812 fetch_fsmap learned FSMap version 9 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x21eed80 con 0x2092ae0 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1167865445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec01e050 con 0x2092ae0 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1167865445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 9) v1 ==== 3108+0+0 (secure 0 0 0) 0x7fb8f40177b0 con 0x2092ae0 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 10 client.4812 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 20 client.4812 populate_metadata read hostname 'smithi033' 2023-10-06T02:45:52.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x22f0e80 0x22f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 4 client.4812 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-10-06T02:45:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 --> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] -- command(tid 0: {"prefix": "status"}) v1 -- 0x220e410 con 0x22f0e80 2023-10-06T02:45:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x22f0e80 0x22f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 10 client.4812 ms_handle_connect on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8fbfff700 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x22f0e80 0x22f1250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb0a6634438782d server_cookie=120a799d9343142f in_seq=0 out_seq=0 2023-10-06T02:45:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 1 -- 172.21.15.33:0/1167865445 <== mds.0 v2:172.21.15.33:6834/2898450531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x220e410 con 0x22f0e80 2023-10-06T02:45:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 10 client.4812 handle_command_reply: tid=0 2023-10-06T02:45:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 client.4812 shutdown 2023-10-06T02:45:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 msgr2=0x22f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 0x22f0400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4015fa0 tx=0x7fb8f4016000 comp rx=0 tx=0).stop 2023-10-06T02:45:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 shutdown_connections 2023-10-06T02:45:52.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:6834/2898450531,v1:172.21.15.33:6835/2898450531] conn(0x22f0e80 0x22f1250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x20dd790 0x22f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x2092ae0 0x22f0400 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 --2- 172.21.15.33:0/1167865445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x2092710 0x22efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 >> 172.21.15.33:0/1167865445 conn(0x2092eb0 msgr2=0x22ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb8ea7fc700 0 client.4812 ms_handle_reset on v2:172.21.15.33:6834/2898450531 2023-10-06T02:45:52.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 shutdown_connections 2023-10-06T02:45:52.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 1 -- 172.21.15.33:0/1167865445 wait complete. 2023-10-06T02:45:52.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.805+0000 7fb904cf5740 20 client.4812 trim_cache size 0 max 16384 2023-10-06T02:45:52.829 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:45:52.829 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8ff12af2-25a8-465d-8914-d29bca58b4dd", 2023-10-06T02:45:52.829 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T02:45:52.829 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4287, 2023-10-06T02:45:52.829 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T02:45:52.830 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T02:45:52.830 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T02:45:52.830 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 29.031324127000001, 2023-10-06T02:45:52.830 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 9, 2023-10-06T02:45:52.830 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 29, 2023-10-06T02:45:52.831 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 29, 2023-10-06T02:45:52.831 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 34.076359113000002 2023-10-06T02:45:52.831 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:45:52.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb8e0060550 msgr2=0x7fb8e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb8e0060550 0x7fb8e0062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009fd0 tx=0x7fb8f400b040 comp rx=0 tx=0).stop 2023-10-06T02:45:52.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 msgr2=0x7fb8fc149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:52.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 0x7fb8fc149fd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000c00 tx=0x7fb8e4007f20 comp rx=0 tx=0).stop 2023-10-06T02:45:52.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 shutdown_connections 2023-10-06T02:45:52.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb8e0060550 0x7fb8e0062a00 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009fd0 tx=0x7fb8f400b040 comp rx=0 tx=0).stop 2023-10-06T02:45:52.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb8fc156100 0x7fb8fc0a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8fc155d30 0x7fb8fc0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 --2- 172.21.15.33:0/1217784371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8fc069580 0x7fb8fc149fd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 >> 172.21.15.33:0/1217784371 conn(0x7fb8fc0af920 msgr2=0x7fb8fc153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 shutdown_connections 2023-10-06T02:45:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:52.809+0000 7fb902a4c700 1 -- 172.21.15.33:0/1217784371 wait complete. 2023-10-06T02:45:52.836 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T02:45:52.847 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T02:45:52.847 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T02:45:52.847 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-06T02:45:52.848 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-06T02:45:52.848 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-06T02:45:52.848 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-06T02:45:52.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:52.848 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:45:52.856 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-06T02:45:52.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:52.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-06T02:45:52.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:52.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-10-06T02:45:52.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:52.986 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T02:45:52.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:52.996 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2023-10-06T02:45:53.061 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2023-10-06T02:45:53.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:45:53.062 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T02:45:53.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:45:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 Processor -- start 2023-10-06T02:45:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 -- start start 2023-10-06T02:45:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1080c9550 0x7fd1080680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd108068630 0x7fd1080c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080c99d0 con 0x7fd108160fd0 2023-10-06T02:45:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd108067050 con 0x7fd1080c9550 2023-10-06T02:45:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10fc41700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd108067330 con 0x7fd108068630 2023-10-06T02:45:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1080c9550 0x7fd1080680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1080c9550 0x7fd1080680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35316/0 (socket says 172.21.15.33:35316) 2023-10-06T02:45:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 -- 172.21.15.33:0/1260769485 learned_addr learned my addr 172.21.15.33:0/1260769485 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10e1de700 1 --2- 172.21.15.33:0/1260769485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd108068630 0x7fd1080c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 -- 172.21.15.33:0/1260769485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd108068630 msgr2=0x7fd1080c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 --2- 172.21.15.33:0/1260769485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd108068630 0x7fd1080c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 -- 172.21.15.33:0/1260769485 >> v1:172.21.15.167:6789/0 conn(0x7fd108160fd0 legacy=0x7fd1081613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10c9db700 1 -- 172.21.15.33:0/1260769485 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881804373 0 0) 0x7fd1080c99d0 con 0x7fd108160fd0 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 -- 172.21.15.33:0/1260769485 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108067610 con 0x7fd1080c9550 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.173+0000 7fd10d1dc700 1 --2- 172.21.15.33:0/1260769485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1080c9550 0x7fd1080680f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0099b0 tx=0x7fd0fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6519595bb20f56e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10c9db700 1 -- 172.21.15.33:0/1260769485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc009ea0 con 0x7fd1080c9550 2023-10-06T02:45:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10c9db700 1 -- 172.21.15.33:0/1260769485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc00b860 con 0x7fd1080c9550 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10c9db700 1 -- 172.21.15.33:0/1260769485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00bc00 con 0x7fd1080c9550 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- 172.21.15.33:0/1260769485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1080c9550 msgr2=0x7fd1080680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1260769485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1080c9550 0x7fd1080680f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0099b0 tx=0x7fd0fc009250 comp rx=0 tx=0).stop 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- 172.21.15.33:0/1260769485 shutdown_connections 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1260769485 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd108068630 0x7fd1080c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1260769485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1080c9550 0x7fd1080680f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- 172.21.15.33:0/1260769485 >> 172.21.15.33:0/1260769485 conn(0x7fd1080c83f0 msgr2=0x7fd1080c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- 172.21.15.33:0/1260769485 shutdown_connections 2023-10-06T02:45:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- 172.21.15.33:0/1260769485 wait complete. 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 Processor -- start 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- start start 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd108068630 0x7fd10816a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1080c9550 0x7fd1080c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 0x7fd1080c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10816e550 con 0x7fd108160fd0 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080af820 con 0x7fd108068630 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.177+0000 7fd10fc41700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080afb00 con 0x7fd1080c9550 2023-10-06T02:45:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 0x7fd1080c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 0x7fd1080c2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35328/0 (socket says 172.21.15.33:35328) 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 -- 172.21.15.33:0/1321631380 learned_addr learned my addr 172.21.15.33:0/1321631380 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10d1dc700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1080c9550 0x7fd1080c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10d9dd700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd108068630 0x7fd10816a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 -- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1080c9550 msgr2=0x7fd1080c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1080c9550 0x7fd1080c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 -- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd108068630 msgr2=0x7fd10816a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd108068630 0x7fd10816a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 -- 172.21.15.33:0/1321631380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc0094f0 con 0x7fd108160fd0 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10e1de700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 0x7fd1080c2200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e8d0 tx=0x7fd1000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100013680 con 0x7fd108160fd0 2023-10-06T02:45:53.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1000137e0 con 0x7fd108160fd0 2023-10-06T02:45:53.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1080afde0 con 0x7fd108160fd0 2023-10-06T02:45:53.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100013ba0 con 0x7fd108160fd0 2023-10-06T02:45:53.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1080b05e0 con 0x7fd108160fd0 2023-10-06T02:45:53.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1080710b0 con 0x7fd108160fd0 2023-10-06T02:45:53.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.181+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd1000123d0 con 0x7fd108160fd0 2023-10-06T02:45:53.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.185+0000 7fd105ffb700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd0f4060550 0x7fd0f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:53.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.185+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd10007cda0 con 0x7fd108160fd0 2023-10-06T02:45:53.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.185+0000 7fd10d9dd700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd0f4060550 0x7fd0f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:53.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.185+0000 7fd10d9dd700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd0f4060550 0x7fd0f4062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8001560 tx=0x7fd0f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:53.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.189+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd10004cf40 con 0x7fd108160fd0 2023-10-06T02:45:53.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.393+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd10815bb80 con 0x7fd108160fd0 2023-10-06T02:45:53.398 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.393+0000 7fd105ffb700 1 -- 172.21.15.33:0/1321631380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd100050560 con 0x7fd108160fd0 2023-10-06T02:45:53.399 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T02:45:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd0f4060550 msgr2=0x7fd0f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd0f4060550 0x7fd0f4062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8001560 tx=0x7fd0f8009040 comp rx=0 tx=0).stop 2023-10-06T02:45:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 msgr2=0x7fd1080c2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 0x7fd1080c2200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e8d0 tx=0x7fd1000091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 shutdown_connections 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd0f4060550 0x7fd0f4062a00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1080c9550 0x7fd1080c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd108160fd0 0x7fd1080c2200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 --2- 172.21.15.33:0/1321631380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd108068630 0x7fd10816a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 >> 172.21.15.33:0/1321631380 conn(0x7fd1080c83f0 msgr2=0x7fd1080691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 shutdown_connections 2023-10-06T02:45:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:53.397+0000 7fd10fc41700 1 -- 172.21.15.33:0/1321631380 wait complete. 2023-10-06T02:45:53.414 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:45:53.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:45:53.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T02:45:53.414 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:45:53.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:45:53.414 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:45:53.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:45:53.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:45:53.415 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:45:53.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.415 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T02:45:53.422 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:53.423 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42559sec preferred_lft 42559sec 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:72:46:44:8b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:45:53.424 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:53.424 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T02:45:53.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T02:45:53.425 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:53.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.467 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:45:53.477 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T02:45:53.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.478 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:45:53.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.530 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:53.530 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:53.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:45:53.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:45:53.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:45:53.530 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:53.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.570 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:45:53.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.578 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-10-06T02:45:53.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.584 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:53.584 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:53.584 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:45:53.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T02:45:53.585 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:53.616 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T02:45:53.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T02:45:53.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T02:45:53.617 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:53.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:53.671 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:53.671 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:53.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T02:45:53.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T02:45:53.671 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:53.699 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T02:45:53.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:53.699 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:45:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:45:53.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:53.707 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:45:53.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:53.773 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:53.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:53.784 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:53.785 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:53.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:53.786 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:53.791 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T02:45:53.792 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T02:45:53.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:53.811 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:53.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:53.866 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:53.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:53.867 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:53.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2023-10-06T02:45:53.869+0000 7f05881f4380 -1 init, newargv = 0x561aa263b7e0 newargc=15 2023-10-06T02:45:53.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[41575]: starting ceph client 2023-10-06T02:45:53.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[41575]: starting fuse 2023-10-06T02:45:54.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:54.993 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:54.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:55.051 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:55.053 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:55.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:55.053 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:55.060 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:45:55.060 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> import re 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2023-10-06T02:45:55.061 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:45:55.062 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> return f 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2023-10-06T02:45:55.063 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:45:55.140 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.41575.asok 2023-10-06T02:45:55.324 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41575.asok 2023-10-06T02:45:55.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:55.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41575.asok status 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T02:45:55.414 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "41575", 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4824, 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4824 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3961952618 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3961952618 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4824 192.168.0.1:0/3961952618", 2023-10-06T02:45:55.416 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/3961952618", 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 9, 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 29, 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2023-10-06T02:45:55.417 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:45:55.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:45:55.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f5744d41700 1 Processor -- start 2023-10-06T02:45:55.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f5744d41700 1 -- start start 2023-10-06T02:45:55.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f5744d41700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57400cf420 0x7f574015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:55.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f5744d41700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400cdf80 con 0x7f5740158d00 2023-10-06T02:45:55.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f5744d41700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574015d730 con 0x7f5740068b60 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f5744d41700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574015da10 con 0x7f57400cf420 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5740158d00 0x7f5740068450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36206/0 (socket says 172.21.15.33:36206) 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57400cf420 0x7f574015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573f7fe700 1 -- 172.21.15.33:0/2415311908 learned_addr learned my addr 172.21.15.33:0/2415311908 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573effd700 1 -- 172.21.15.33:0/2415311908 >> v1:172.21.15.167:6789/0 conn(0x7f5740068b60 legacy=0x7f57401590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573effd700 1 -- 172.21.15.33:0/2415311908 >> v1:172.21.15.33:6789/0 conn(0x7f5740158d00 legacy=0x7f5740068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573effd700 1 -- 172.21.15.33:0/2415311908 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400cd930 con 0x7f57400cf420 2023-10-06T02:45:55.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573effd700 1 --2- 172.21.15.33:0/2415311908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57400cf420 0x7f574015ca70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5728009e60 tx=0x7f5728009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9485a9ef576db7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573e7fc700 1 -- 172.21.15.33:0/2415311908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800e070 con 0x7f57400cf420 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573e7fc700 1 -- 172.21.15.33:0/2415311908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572800bc30 con 0x7f57400cf420 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.525+0000 7f573e7fc700 1 -- 172.21.15.33:0/2415311908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728012430 con 0x7f57400cf420 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 -- 172.21.15.33:0/2415311908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57400cf420 msgr2=0x7f574015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 --2- 172.21.15.33:0/2415311908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57400cf420 0x7f574015ca70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5728009e60 tx=0x7f5728009780 comp rx=0 tx=0).stop 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 -- 172.21.15.33:0/2415311908 shutdown_connections 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 --2- 172.21.15.33:0/2415311908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57400cf420 0x7f574015ca70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 -- 172.21.15.33:0/2415311908 >> 172.21.15.33:0/2415311908 conn(0x7f5740067a80 msgr2=0x7f57400cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:55.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 -- 172.21.15.33:0/2415311908 shutdown_connections 2023-10-06T02:45:55.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 -- 172.21.15.33:0/2415311908 wait complete. 2023-10-06T02:45:55.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 Processor -- start 2023-10-06T02:45:55.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.529+0000 7f5744d41700 1 -- start start 2023-10-06T02:45:55.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 0x7f57401498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57400cf420 0x7f57400b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5740158d00 0x7f57401521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5740149e00 con 0x7f5740068b60 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400b3ac0 con 0x7f57400cf420 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400b2eb0 con 0x7f5740158d00 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 0x7f57401498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 0x7f57401498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35356/0 (socket says 172.21.15.33:35356) 2023-10-06T02:45:55.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 -- 172.21.15.33:0/1551271037 learned_addr learned my addr 172.21.15.33:0/1551271037 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573ffff700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5740158d00 0x7f57401521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573effd700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57400cf420 0x7f57400b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 -- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5740158d00 msgr2=0x7f57401521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5740158d00 0x7f57401521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 -- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57400cf420 msgr2=0x7f57400b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57400cf420 0x7f57400b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 -- 172.21.15.33:0/1551271037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728009a20 con 0x7f5740068b60 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f573f7fe700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 0x7f57401498c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f573000c560 tx=0x7f5730001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730005b30 con 0x7f5740068b60 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730005cb0 con 0x7f5740068b60 2023-10-06T02:45:55.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57400b3190 con 0x7f5740068b60 2023-10-06T02:45:55.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730007320 con 0x7f5740068b60 2023-10-06T02:45:55.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57400b3750 con 0x7f5740068b60 2023-10-06T02:45:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.533+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5740071040 con 0x7f5740068b60 2023-10-06T02:45:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.537+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f573001b3b0 con 0x7f5740068b60 2023-10-06T02:45:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.537+0000 7f57377fe700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5720060550 0x7f5720062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.537+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f573007c1f0 con 0x7f5740068b60 2023-10-06T02:45:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.537+0000 7f573effd700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5720060550 0x7f5720062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.537+0000 7f573effd700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5720060550 0x7f5720062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5740088f70 tx=0x7f5728019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.541+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f573004c2f0 con 0x7f5740068b60 2023-10-06T02:45:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57400cdc10 con 0x7f5740068b60 2023-10-06T02:45:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f57377fe700 1 -- 172.21.15.33:0/1551271037 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f573004f910 con 0x7f5740068b60 2023-10-06T02:45:55.750 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5720060550 msgr2=0x7f5720062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5720060550 0x7f5720062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5740088f70 tx=0x7f5728019000 comp rx=0 tx=0).stop 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 msgr2=0x7f57401498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 0x7f57401498c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f573000c560 tx=0x7f5730001020 comp rx=0 tx=0).stop 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 shutdown_connections 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5720060550 0x7f5720062a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5740158d00 0x7f57401521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57400cf420 0x7f57400b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 --2- 172.21.15.33:0/1551271037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740068b60 0x7f57401498c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 >> 172.21.15.33:0/1551271037 conn(0x7f5740067a80 msgr2=0x7f57400b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 shutdown_connections 2023-10-06T02:45:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:55.745+0000 7f5744d41700 1 -- 172.21.15.33:0/1551271037 wait complete. 2023-10-06T02:45:55.764 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:45:55.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:45:55.765 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T02:45:55.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.765 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:55.776 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:45:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T02:45:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42556sec preferred_lft 42556sec 2023-10-06T02:45:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T02:45:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.777 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:55.777 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:45:55.778 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:72:46:44:8b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:55.778 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:45:55.778 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.778 INFO:teuthology.orchestra.run.smithi033.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:45:55.778 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ee:c8:8e:78:96:20 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:55.779 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:45:55.779 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.779 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::5ca2:30ff:feae:fa92/64 scope link 2023-10-06T02:45:55.779 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.779 INFO:teuthology.orchestra.run.smithi033.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:45:55.779 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ee:c8:8e:78:96:20 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:45:55.780 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ecc8:8eff:fe78:9620/64 scope link 2023-10-06T02:45:55.780 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:55.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.780 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:45:55.795 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T02:45:55.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.797 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T02:45:55.848 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T02:45:55.848 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:45:55.848 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T02:45:55.849 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:45:55.849 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T02:45:55.849 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:45:55.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.849 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:55.849 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:55.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:45:55.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:45:55.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:45:55.850 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:55.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.892 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:45:55.899 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:45:55.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.900 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2023-10-06T02:45:55.947 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T02:45:55.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.948 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:55.948 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:55.948 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:45:55.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T02:45:55.949 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:55.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:55.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 9a:f5:69:ee:9a:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:45:56.041 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:45:56.042 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.042 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::98f5:69ff:feee:9a9b/64 scope link 2023-10-06T02:45:56.042 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:56.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T02:45:56.055 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:45:56.055 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:45:56.055 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 9a:f5:69:ee:9a:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::98f5:69ff:feee:9a9b/64 scope link 2023-10-06T02:45:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:56.057 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T02:45:56.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T02:45:56.058 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T02:45:56.059 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:56.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:56.119 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:45:56.119 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:45:56.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.1 up 2023-10-06T02:45:56.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T02:45:56.119 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:45:56.146 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-06T02:45:56.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:56.147 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T02:45:56.195 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T02:45:56.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:56.196 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T02:45:56.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:56.218 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:56.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:56.233 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:56.234 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:56.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:56.235 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:56.241 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:45:56.241 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-06T02:45:56.241 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-06T02:45:56.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:56.259 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:56.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:56.315 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:56.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:56.316 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:56.321 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:45:56.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi033.stderr:2023-10-06T02:45:56.317+0000 7fb962e76380 -1 init, newargv = 0x563ec8327ad0 newargc=15 2023-10-06T02:45:56.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi033.stderr:ceph-fuse[41716]: starting ceph client 2023-10-06T02:45:56.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi033.stderr:ceph-fuse[41716]: starting fuse 2023-10-06T02:45:57.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:45:57.441 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:57.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:57.499 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:57.500 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:57.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:57.501 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:45:57.508 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:45:57.508 INFO:teuthology.orchestra.run.smithi033.stdout:53 2023-10-06T02:45:57.508 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> import re 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:57.509 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> return f 2023-10-06T02:45:57.510 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:45:57.511 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:45:57.511 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.2")) 2023-10-06T02:45:57.511 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:45:57.598 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.2.41716.asok 2023-10-06T02:45:57.781 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.41716.asok 2023-10-06T02:45:57.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:57.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.41716.asok status 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "2", 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2023-10-06T02:45:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "41716", 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4842, 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4842 2023-10-06T02:45:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.2:0", 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1386243014 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.2:0", 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1386243014 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:45:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4842 192.168.0.2:0/1386243014", 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.2:0/1386243014", 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 9, 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 29, 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2023-10-06T02:45:57.880 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:45:57.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:45:57.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79fd106700 1 Processor -- start 2023-10-06T02:45:57.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79fd106700 1 -- start start 2023-10-06T02:45:57.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79fd106700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79f80c93a0 0x7f79f8066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:57.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79fd106700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f806b420 con 0x7f79f806b050 2023-10-06T02:45:57.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79fd106700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f806a4e0 con 0x7f79f80c67e0 2023-10-06T02:45:58.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79fd106700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80674f0 con 0x7f79f80c93a0 2023-10-06T02:45:58.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79effff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79f80c93a0 0x7f79f8066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:58.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79f7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f79f806b050 0x7f79f80c60d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36222/0 (socket says 172.21.15.33:36222) 2023-10-06T02:45:58.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79f7fff700 1 -- 172.21.15.33:0/1796054232 learned_addr learned my addr 172.21.15.33:0/1796054232 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:45:58.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79effff700 1 -- 172.21.15.33:0/1796054232 >> v1:172.21.15.167:6789/0 conn(0x7f79f80c67e0 legacy=0x7f79f8166c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79effff700 1 -- 172.21.15.33:0/1796054232 >> v1:172.21.15.33:6789/0 conn(0x7f79f806b050 legacy=0x7f79f80c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.993+0000 7f79f77fe700 1 -- 172.21.15.33:0/1796054232 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755695980 0 0) 0x7f79f806b420 con 0x7f79f806b050 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79effff700 1 -- 172.21.15.33:0/1796054232 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f8067700 con 0x7f79f80c93a0 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79effff700 1 --2- 172.21.15.33:0/1796054232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c93a0 0x7f79f8066fb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098e0 tx=0x7f79e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27725517e8988cb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79f77fe700 1 -- 172.21.15.33:0/1796054232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0009e80 con 0x7f79f80c93a0 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79f77fe700 1 -- 172.21.15.33:0/1796054232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b860 con 0x7f79f80c93a0 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79f77fe700 1 -- 172.21.15.33:0/1796054232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000bbe0 con 0x7f79f80c93a0 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 -- 172.21.15.33:0/1796054232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c93a0 msgr2=0x7f79f8066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 --2- 172.21.15.33:0/1796054232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c93a0 0x7f79f8066fb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098e0 tx=0x7f79e00091a0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 -- 172.21.15.33:0/1796054232 shutdown_connections 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 --2- 172.21.15.33:0/1796054232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c93a0 0x7f79f8066fb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 -- 172.21.15.33:0/1796054232 >> 172.21.15.33:0/1796054232 conn(0x7f79f80c8240 msgr2=0x7f79f80c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 -- 172.21.15.33:0/1796054232 shutdown_connections 2023-10-06T02:45:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 -- 172.21.15.33:0/1796054232 wait complete. 2023-10-06T02:45:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 Processor -- start 2023-10-06T02:45:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:57.997+0000 7f79fd106700 1 -- start start 2023-10-06T02:45:58.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79f806b050 0x7f79f8153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:58.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 0x7f79f80b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:58.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f80c93a0 0x7f79f81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:58.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80b0500 con 0x7f79f80c67e0 2023-10-06T02:45:58.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80b1010 con 0x7f79f80c93a0 2023-10-06T02:45:58.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80b0070 con 0x7f79f806b050 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 0x7f79f80b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 0x7f79f80b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35378/0 (socket says 172.21.15.33:35378) 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 -- 172.21.15.33:0/3972125470 learned_addr learned my addr 172.21.15.33:0/3972125470 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79f7fff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79f806b050 0x7f79f8153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fc905700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f80c93a0 0x7f79f81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 -- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79f806b050 msgr2=0x7f79f8153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79f806b050 0x7f79f8153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 -- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f80c93a0 msgr2=0x7f79f81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f80c93a0 0x7f79f81531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 -- 172.21.15.33:0/3972125470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009420 con 0x7f79f80c67e0 2023-10-06T02:45:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79f7fff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79f806b050 0x7f79f8153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fc905700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f80c93a0 0x7f79f81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79effff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 0x7f79f80b0930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79f806cfc0 tx=0x7f79e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e001f070 con 0x7f79f80c67e0 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b9d0 con 0x7f79f80c67e0 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f8153710 con 0x7f79f80c67e0 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0025670 con 0x7f79f80c67e0 2023-10-06T02:45:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f80af740 con 0x7f79f80c67e0 2023-10-06T02:45:58.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.001+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f80b0220 con 0x7f79f80c67e0 2023-10-06T02:45:58.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.005+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f79e0023070 con 0x7f79f80c67e0 2023-10-06T02:45:58.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.005+0000 7f79f4ff9700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f79d8060550 0x7f79d8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:45:58.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.005+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f79e007e380 con 0x7f79f80c67e0 2023-10-06T02:45:58.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.005+0000 7f79f7fff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f79d8060550 0x7f79d8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:45:58.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.005+0000 7f79f7fff700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f79d8060550 0x7f79d8062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000c00 tx=0x7f79e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:45:58.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.009+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f79f80b0220 con 0x7f79f80c67e0 2023-10-06T02:45:58.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.209+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79f80b01e0 con 0x7f79f80c67e0 2023-10-06T02:45:58.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.209+0000 7f79f4ff9700 1 -- 172.21.15.33:0/3972125470 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f79e0015070 con 0x7f79f80c67e0 2023-10-06T02:45:58.214 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f79d8060550 msgr2=0x7f79d8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f79d8060550 0x7f79d8062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000c00 tx=0x7f79e8009040 comp rx=0 tx=0).stop 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 msgr2=0x7f79f80b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 0x7f79f80b0930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79f806cfc0 tx=0x7f79e0017000 comp rx=0 tx=0).stop 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 shutdown_connections 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f80c93a0 0x7f79f81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79f80c67e0 0x7f79f80b0930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f79d8060550 0x7f79d8062a00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 --2- 172.21.15.33:0/3972125470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79f806b050 0x7f79f8153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:45:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 >> 172.21.15.33:0/3972125470 conn(0x7f79f80c8240 msgr2=0x7f79f8155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:45:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 shutdown_connections 2023-10-06T02:45:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:45:58.213+0000 7f79fd106700 1 -- 172.21.15.33:0/3972125470 wait complete. 2023-10-06T02:45:58.228 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:45:58.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:45:58.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T02:45:58.228 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T02:45:58.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T02:45:58.228 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:45:58.229 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:45:58.229 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:45:58.229 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:45:58.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.229 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T02:45:58.237 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:45:58.237 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:45:58.237 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:45:58.237 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:58.237 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T02:45:58.237 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42568sec preferred_lft 42568sec 2023-10-06T02:45:58.238 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T02:45:58.239 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:58.239 INFO:teuthology.orchestra.run.smithi167.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:45:58.239 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 02:42:ef:ad:7a:9e brd ff:ff:ff:ff:ff:ff 2023-10-06T02:45:58.239 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:45:58.239 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:45:58.239 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T02:45:58.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.239 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:45:58.239 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:45:58.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2023-10-06T02:45:58.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2023-10-06T02:45:58.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2023-10-06T02:45:58.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T02:45:58.240 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:45:58.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.282 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:45:58.343 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T02:45:58.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.345 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T02:45:58.393 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T02:45:58.393 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:45:58.393 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-06T02:45:58.393 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:45:58.393 INFO:teuthology.orchestra.run.smithi167.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:45:58.393 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-06T02:45:58.394 INFO:teuthology.orchestra.run.smithi167.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-06T02:45:58.394 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:45:58.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.394 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:45:58.394 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:45:58.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:45:58.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:45:58.395 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:45:58.395 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:45:58.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.435 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T02:45:58.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.442 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2023-10-06T02:45:58.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.491 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:45:58.492 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:45:58.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:45:58.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T02:45:58.492 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:45:58.525 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T02:45:58.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T02:45:58.525 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T02:45:58.526 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:45:58.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:45:58.592 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:45:58.592 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:45:58.592 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2023-10-06T02:45:58.592 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T02:45:58.592 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:45:58.621 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T02:45:58.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:58.622 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T02:45:58.638 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T02:45:58.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:45:58.638 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:45:58.643 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2023-10-06T02:45:58.658 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:58.658 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:58.716 INFO:teuthology.orchestra.run.smithi167.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:58.717 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:58.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:58.717 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2023-10-06T02:45:58.724 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T02:45:58.725 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T02:45:58.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2023-10-06T02:45:58.743 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:58.743 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:58.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi167.stderr:2023-10-06T02:45:58.799+0000 7f1967321380 -1 init, newargv = 0x5639d1d72820 newargc=15 2023-10-06T02:45:58.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi167.stderr:ceph-fuse[39030]: starting ceph client 2023-10-06T02:45:58.804 INFO:teuthology.orchestra.run.smithi167.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:58.805 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:58.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:58.806 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2023-10-06T02:45:58.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi167.stderr:ceph-fuse[39030]: starting fuse 2023-10-06T02:45:59.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2023-10-06T02:45:59.930 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:45:59.931 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:45:59.940 INFO:teuthology.orchestra.run.smithi167.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:45:59.941 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:45:59.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:45:59.942 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2023-10-06T02:45:59.949 INFO:teuthology.orchestra.run.smithi167.stdout:52 2023-10-06T02:45:59.949 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> import glob 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> import re 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> import os 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> import subprocess 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> def _find_admin_socket(client_name): 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> files = glob.glob(asok_path) 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T02:45:59.950 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> # Given a non-glob path, it better be there 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> if "*" not in asok_path: 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> assert(len(files) == 1) 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> return files[0] 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> for f in files: 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> contents = proc_f.read() 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> if mountpoint in contents: 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> return f 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:45:59.951 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:45:59.952 DEBUG:teuthology.orchestra.run.smithi167:> print(_find_admin_socket("client.1")) 2023-10-06T02:45:59.952 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T02:46:00.076 INFO:teuthology.orchestra.run.smithi167.stdout:/var/run/ceph/ceph-client.1.39030.asok 2023-10-06T02:46:00.263 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39030.asok 2023-10-06T02:46:00.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:00.263 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39030.asok status 2023-10-06T02:46:00.349 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T02:46:00.349 INFO:teuthology.orchestra.run.smithi167.stdout: "metadata": { 2023-10-06T02:46:00.349 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:46:00.349 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:46:00.349 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "1", 2023-10-06T02:46:00.349 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2023-10-06T02:46:00.350 INFO:teuthology.orchestra.run.smithi167.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T02:46:00.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pid": "39030", 2023-10-06T02:46:00.350 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2023-10-06T02:46:00.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:00.350 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_count": 0, 2023-10-06T02:46:00.350 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_pinned_count": 0, 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4637, 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": { 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 4637 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 3964399284 2023-10-06T02:46:00.351 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T02:46:00.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:00.352 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2023-10-06T02:46:00.352 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2023-10-06T02:46:00.352 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:46:00.352 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 3964399284 2023-10-06T02:46:00.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "inst_str": "client.4637 192.168.0.1:0/3964399284", 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "addr_str": "192.168.0.1:0/3964399284", 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "inode_count": 1, 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_epoch": 9, 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch": 29, 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "blocklisted": false, 2023-10-06T02:46:00.353 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs" 2023-10-06T02:46:00.354 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T02:46:00.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:46:00.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 Processor -- start 2023-10-06T02:46:00.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 -- start start 2023-10-06T02:46:00.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 0x7f946015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:00.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f946015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:00.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946015d830 con 0x7f9460158bf0 2023-10-06T02:46:00.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9460067c10 con 0x7f94600b0530 2023-10-06T02:46:00.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9466c69700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b00a0 con 0x7f9460158fc0 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f945ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 0x7f946015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f946015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f946015cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:59052/0 (socket says 172.21.15.33:59052) 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f945ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 0x7f946015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44630/0 (socket says 172.21.15.33:44630) 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 -- 172.21.15.33:0/3503036418 learned_addr learned my addr 172.21.15.33:0/3503036418 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 -- 172.21.15.33:0/3503036418 >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 msgr2=0x7f946015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 --2- 172.21.15.33:0/3503036418 >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 0x7f946015ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 -- 172.21.15.33:0/3503036418 >> v1:172.21.15.167:6789/0 conn(0x7f9460158bf0 legacy=0x7f94600b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 -- 172.21.15.33:0/3503036418 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600cba20 con 0x7f9460158fc0 2023-10-06T02:46:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f945ffff700 1 --2- 172.21.15.33:0/3503036418 >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 0x7f946015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f9465206700 1 --2- 172.21.15.33:0/3503036418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f946015cfb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94580099b0 tx=0x7f9458009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d95ec90a57f4e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f945f7fe700 1 -- 172.21.15.33:0/3503036418 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458009ea0 con 0x7f9460158fc0 2023-10-06T02:46:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f945f7fe700 1 -- 172.21.15.33:0/3503036418 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800b860 con 0x7f9460158fc0 2023-10-06T02:46:00.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.469+0000 7f945f7fe700 1 -- 172.21.15.33:0/3503036418 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800bc00 con 0x7f9460158fc0 2023-10-06T02:46:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- 172.21.15.33:0/3503036418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 msgr2=0x7f946015cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 --2- 172.21.15.33:0/3503036418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f946015cfb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94580099b0 tx=0x7f9458009250 comp rx=0 tx=0).stop 2023-10-06T02:46:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- 172.21.15.33:0/3503036418 shutdown_connections 2023-10-06T02:46:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 --2- 172.21.15.33:0/3503036418 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f946015cfb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 --2- 172.21.15.33:0/3503036418 >> v2:172.21.15.167:3300/0 conn(0x7f94600b0530 0x7f946015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- 172.21.15.33:0/3503036418 >> 172.21.15.33:0/3503036418 conn(0x7f94600cfd70 msgr2=0x7f94600d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- 172.21.15.33:0/3503036418 shutdown_connections 2023-10-06T02:46:00.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- 172.21.15.33:0/3503036418 wait complete. 2023-10-06T02:46:00.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 Processor -- start 2023-10-06T02:46:00.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- start start 2023-10-06T02:46:00.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9460158bf0 0x7f9460159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:00.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f9460159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f946015a500 0x7f946015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460169730 con 0x7f946015a500 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460169a10 con 0x7f9460158bf0 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9466c69700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9460169cf0 con 0x7f9460158fc0 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f945ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f9460159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.473+0000 7f9465206700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f946015a500 0x7f946015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f9460159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:59058/0 (socket says 172.21.15.33:59058) 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 -- 172.21.15.33:0/2184010483 learned_addr learned my addr 172.21.15.33:0/2184010483 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f9464a05700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9460158bf0 0x7f9460159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 -- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9460158bf0 msgr2=0x7f9460159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:00.483 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9460158bf0 0x7f9460159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 -- 172.21.15.33:0/2184010483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f946015a500 msgr2=0x7f946015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f946015a500 0x7f946015b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 -- 172.21.15.33:0/2184010483 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448009a20 con 0x7f9460158fc0 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f9465206700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f946015a500 0x7f946015b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945ffff700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f9460159fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f944800c0c0 tx=0x7f944800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800cc00 con 0x7f9460158fc0 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944800f040 con 0x7f9460158fc0 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448014480 con 0x7f9460158fc0 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94580094f0 con 0x7f9460158fc0 2023-10-06T02:46:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9460169430 con 0x7f9460158fc0 2023-10-06T02:46:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.477+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f94480147f0 con 0x7f9460158fc0 2023-10-06T02:46:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.481+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946016ca70 con 0x7f9460158fc0 2023-10-06T02:46:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.481+0000 7f945cff9700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9440060550 0x7f9440062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.481+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f944807d9d0 con 0x7f9460158fc0 2023-10-06T02:46:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.481+0000 7f9464a05700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9440060550 0x7f9440062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.481+0000 7f9464a05700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9440060550 0x7f9440062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9460157430 tx=0x7f9450009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.493+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f946016ca70 con 0x7f9460158fc0 2023-10-06T02:46:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.693+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f946016ca70 con 0x7f9460158fc0 2023-10-06T02:46:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.693+0000 7f945cff9700 1 -- 172.21.15.33:0/2184010483 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f946016ca70 con 0x7f9460158fc0 2023-10-06T02:46:00.698 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-06T02:46:00.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9440060550 msgr2=0x7f9440062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:00.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9440060550 0x7f9440062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9460157430 tx=0x7f9450009040 comp rx=0 tx=0).stop 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 msgr2=0x7f9460159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f9460159fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f944800c0c0 tx=0x7f944800b040 comp rx=0 tx=0).stop 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 shutdown_connections 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9440060550 0x7f9440062a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9460158fc0 0x7f9460159fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f946015a500 0x7f946015b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 --2- 172.21.15.33:0/2184010483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9460158bf0 0x7f9460159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 >> 172.21.15.33:0/2184010483 conn(0x7f94600cfd70 msgr2=0x7f94600b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 shutdown_connections 2023-10-06T02:46:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:00.697+0000 7f9466c69700 1 -- 172.21.15.33:0/2184010483 wait complete. 2023-10-06T02:46:00.718 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:46:00.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:46:00.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2023-10-06T02:46:00.719 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T02:46:00.719 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:00.719 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:46:00.719 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:46:00.719 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:46:00.719 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T02:46:00.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.719 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2023-10-06T02:46:00.728 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:46:00.728 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:46:00.728 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:46:00.728 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.728 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:00.729 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42565sec preferred_lft 42565sec 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:46:00.730 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 02:42:ef:ad:7a:9e brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 02:55:cc:ec:51:97 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::c8:c4ff:fe66:e4d4/64 scope link 2023-10-06T02:46:00.731 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.732 INFO:teuthology.orchestra.run.smithi167.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:46:00.732 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 02:55:cc:ec:51:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:46:00.732 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::55:ccff:feec:5197/64 scope link 2023-10-06T02:46:00.732 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.732 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:46:00.784 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-06T02:46:00.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.787 DEBUG:teuthology.orchestra.run.smithi167:> route 2023-10-06T02:46:00.796 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2023-10-06T02:46:00.796 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:46:00.796 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-06T02:46:00.796 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T02:46:00.797 INFO:teuthology.orchestra.run.smithi167.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:46:00.797 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-06T02:46:00.797 INFO:teuthology.orchestra.run.smithi167.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-06T02:46:00.797 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:46:00.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.797 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:46:00.797 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:46:00.797 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T02:46:00.797 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T02:46:00.798 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T02:46:00.798 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:46:00.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.839 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T02:46:00.888 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:46:00.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.889 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2023-10-06T02:46:00.896 INFO:teuthology.orchestra.run.smithi167.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T02:46:00.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.897 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:46:00.897 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:46:00.897 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:46:00.898 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T02:46:00.898 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:46:00.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.931 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T02:46:00.993 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 22:13:71:bf:2a:41 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.994 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::2013:71ff:febf:2a41/64 scope link 2023-10-06T02:46:00.995 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:00.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:00.996 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T02:46:01.008 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 22:13:71:bf:2a:41 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::2013:71ff:febf:2a41/64 scope link 2023-10-06T02:46:01.009 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:01.010 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T02:46:01.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T02:46:01.011 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:46:01.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:01.074 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec bash -c ' 2023-10-06T02:46:01.074 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2023-10-06T02:46:01.074 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.1 up 2023-10-06T02:46:01.074 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T02:46:01.074 DEBUG:teuthology.orchestra.run.smithi167:> ') 2023-10-06T02:46:01.101 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-06T02:46:01.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:46:01.102 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:01.151 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T02:46:01.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:46:01.152 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:01.158 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2023-10-06T02:46:01.173 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:01.173 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:01.232 INFO:teuthology.orchestra.run.smithi167.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:01.232 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:01.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:01.233 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2023-10-06T02:46:01.239 INFO:teuthology.orchestra.run.smithi167.stdout:52 2023-10-06T02:46:01.240 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-06T02:46:01.240 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-06T02:46:01.242 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2023-10-06T02:46:01.256 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:01.256 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:01.310 INFO:teuthology.orchestra.run.smithi167.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:01.311 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:01.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:01.311 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2023-10-06T02:46:01.318 INFO:teuthology.orchestra.run.smithi167.stdout:52 2023-10-06T02:46:01.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi167.stderr:2023-10-06T02:46:01.323+0000 7fd1450a8380 -1 init, newargv = 0x5602f82adc70 newargc=15 2023-10-06T02:46:01.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi167.stderr:ceph-fuse[39144]: starting ceph client 2023-10-06T02:46:01.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi167.stderr:ceph-fuse[39144]: starting fuse 2023-10-06T02:46:02.420 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2023-10-06T02:46:02.440 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:02.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:02.497 INFO:teuthology.orchestra.run.smithi167.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:02.498 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:02.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.500 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2023-10-06T02:46:02.504 INFO:teuthology.orchestra.run.smithi167.stdout:52 2023-10-06T02:46:02.504 INFO:teuthology.orchestra.run.smithi167.stdout:53 2023-10-06T02:46:02.505 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> import glob 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> import re 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> import os 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> import subprocess 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> def _find_admin_socket(client_name): 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> files = glob.glob(asok_path) 2023-10-06T02:46:02.505 DEBUG:teuthology.orchestra.run.smithi167:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> # Given a non-glob path, it better be there 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> if "*" not in asok_path: 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> assert(len(files) == 1) 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> return files[0] 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> for f in files: 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> contents = proc_f.read() 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> if mountpoint in contents: 2023-10-06T02:46:02.506 DEBUG:teuthology.orchestra.run.smithi167:> return f 2023-10-06T02:46:02.507 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:46:02.507 DEBUG:teuthology.orchestra.run.smithi167:> 2023-10-06T02:46:02.507 DEBUG:teuthology.orchestra.run.smithi167:> print(_find_admin_socket("client.3")) 2023-10-06T02:46:02.507 DEBUG:teuthology.orchestra.run.smithi167:> ' 2023-10-06T02:46:02.631 INFO:teuthology.orchestra.run.smithi167.stdout:/var/run/ceph/ceph-client.3.39144.asok 2023-10-06T02:46:02.816 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39144.asok 2023-10-06T02:46:02.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.816 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39144.asok status 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout: "metadata": { 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "3", 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2023-10-06T02:46:02.907 INFO:teuthology.orchestra.run.smithi167.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "pid": "39144", 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_count": 0, 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_pinned_count": 0, 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4649, 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": { 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2023-10-06T02:46:02.908 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 4649 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.2:0", 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 300262292 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2023-10-06T02:46:02.909 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.2:0", 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 300262292 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "inst_str": "client.4649 192.168.0.2:0/300262292", 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "addr_str": "192.168.0.2:0/300262292", 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "inode_count": 1, 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_epoch": 9, 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch": 29, 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "blocklisted": false, 2023-10-06T02:46:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs" 2023-10-06T02:46:02.911 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T02:46:02.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.916 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2023-10-06T02:46:02.916 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:02.925 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2023-10-06T02:46:02.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:02.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:02.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.940 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2023-10-06T02:46:02.940 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-06T02:46:02.946 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2023-10-06T02:46:02.946 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-06T02:46:02.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T02:46:02.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.958 DEBUG:teuthology.orchestra.run.smithi167:> stat --file-system '--printf=%T 2023-10-06T02:46:02.958 DEBUG:teuthology.orchestra.run.smithi167:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T02:46:02.966 INFO:teuthology.orchestra.run.smithi167.stdout:fuseblk 2023-10-06T02:46:02.966 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T02:46:02.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:02.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T02:46:03.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:03.024 DEBUG:teuthology.orchestra.run.smithi167:> stat --file-system '--printf=%T 2023-10-06T02:46:03.025 DEBUG:teuthology.orchestra.run.smithi167:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:03.033 INFO:teuthology.orchestra.run.smithi167.stdout:fuseblk 2023-10-06T02:46:03.033 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:03.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:03.033 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:03.047 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T02:46:03.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 Processor -- start 2023-10-06T02:46:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 -- start start 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6230067970 0x7f623015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6230157eb0 con 0x7f62300adb60 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6230069820 con 0x7f6230067970 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623614f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6230067ea0 con 0x7f62300adf30 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623514d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f62300adb60 0x7f62300c6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39000/0 (socket says 172.21.15.33:39000) 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623494c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6230067970 0x7f623015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.177+0000 7f623514d700 1 -- 172.21.15.33:0/946213331 learned_addr learned my addr 172.21.15.33:0/946213331 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623594e700 1 --2- 172.21.15.33:0/946213331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623494c700 1 -- 172.21.15.33:0/946213331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 msgr2=0x7f623015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f622ffff700 1 -- 172.21.15.33:0/946213331 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636353522 0 0) 0x7f6220001110 con 0x7f62300adb60 2023-10-06T02:46:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623494c700 1 --2- 172.21.15.33:0/946213331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623015ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623494c700 1 -- 172.21.15.33:0/946213331 >> v1:172.21.15.33:6789/0 conn(0x7f62300adb60 legacy=0x7f62300c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623494c700 1 -- 172.21.15.33:0/946213331 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230068180 con 0x7f6230067970 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623494c700 1 --2- 172.21.15.33:0/946213331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623015a5d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f621c009e60 tx=0x7f621c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a670e49be33aef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f622ffff700 1 -- 172.21.15.33:0/946213331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00e070 con 0x7f6230067970 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f622ffff700 1 -- 172.21.15.33:0/946213331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c00bc30 con 0x7f6230067970 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f622ffff700 1 -- 172.21.15.33:0/946213331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c012430 con 0x7f6230067970 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 -- 172.21.15.33:0/946213331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 msgr2=0x7f623015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 --2- 172.21.15.33:0/946213331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623015a5d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f621c009e60 tx=0x7f621c009780 comp rx=0 tx=0).stop 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 -- 172.21.15.33:0/946213331 shutdown_connections 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 --2- 172.21.15.33:0/946213331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 --2- 172.21.15.33:0/946213331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623015a5d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 -- 172.21.15.33:0/946213331 >> 172.21.15.33:0/946213331 conn(0x7f62300c8240 msgr2=0x7f62300c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 -- 172.21.15.33:0/946213331 shutdown_connections 2023-10-06T02:46:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 -- 172.21.15.33:0/946213331 wait complete. 2023-10-06T02:46:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 Processor -- start 2023-10-06T02:46:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.181+0000 7f623614f700 1 -- start start 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62300adb60 0x7f6230166d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6230152a70 con 0x7f6230067970 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6230167690 con 0x7f62300adb60 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6230167970 con 0x7f62300adf30 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623014b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54014/0 (socket says 172.21.15.33:54014) 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 -- 172.21.15.33:0/2795720251 learned_addr learned my addr 172.21.15.33:0/2795720251 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 -- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 msgr2=0x7f623014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623594e700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623494c700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62300adb60 0x7f6230166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623014bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 -- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62300adb60 msgr2=0x7f6230166d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62300adb60 0x7f6230166d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 -- 172.21.15.33:0/2795720251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009a20 con 0x7f6230067970 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623594e700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623494c700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62300adb60 0x7f6230166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623514d700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623014b2d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f622000c7e0 tx=0x7f6220007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220005b30 con 0x7f6230067970 2023-10-06T02:46:03.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6220005c90 con 0x7f6230067970 2023-10-06T02:46:03.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6230167c50 con 0x7f6230067970 2023-10-06T02:46:03.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220007320 con 0x7f6230067970 2023-10-06T02:46:03.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.185+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623014b810 con 0x7f6230067970 2023-10-06T02:46:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.189+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f622001b400 con 0x7f6230067970 2023-10-06T02:46:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.189+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6230071040 con 0x7f6230067970 2023-10-06T02:46:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.189+0000 7f622d7fa700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6218060550 0x7f6218062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.189+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f622007c580 con 0x7f6230067970 2023-10-06T02:46:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.189+0000 7f623494c700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6218060550 0x7f6218062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.189+0000 7f623494c700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6218060550 0x7f6218062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f621c009e60 tx=0x7f621c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.201+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6220050230 con 0x7f6230067970 2023-10-06T02:46:03.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.369+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6230157e80 con 0x7f6230067970 2023-10-06T02:46:03.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f622d7fa700 1 -- 172.21.15.33:0/2795720251 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f622004fdc0 con 0x7f6230067970 2023-10-06T02:46:03.378 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:03.378 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:03.329988+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6218060550 msgr2=0x7f6218062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6218060550 0x7f6218062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f621c009e60 tx=0x7f621c019000 comp rx=0 tx=0).stop 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 msgr2=0x7f623014b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623014b2d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f622000c7e0 tx=0x7f6220007c50 comp rx=0 tx=0).stop 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 shutdown_connections 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6218060550 0x7f6218062a00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62300adf30 0x7f623014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62300adb60 0x7f6230166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 --2- 172.21.15.33:0/2795720251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6230067970 0x7f623014b2d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 >> 172.21.15.33:0/2795720251 conn(0x7f62300c8240 msgr2=0x7f6230159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:03.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 shutdown_connections 2023-10-06T02:46:03.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.373+0000 7f623614f700 1 -- 172.21.15.33:0/2795720251 wait complete. 2023-10-06T02:46:03.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:46:03.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:03.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f3995308700 1 Processor -- start 2023-10-06T02:46:03.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f3995308700 1 -- start start 2023-10-06T02:46:03.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f3995308700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3990158c60 0x7f3990068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f3995308700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399015d840 con 0x7f39900cf7c0 2023-10-06T02:46:03.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f3995308700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39900ce580 con 0x7f3990159030 2023-10-06T02:46:03.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f3995308700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39900cf620 con 0x7f3990158c60 2023-10-06T02:46:03.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f398ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3990158c60 0x7f3990068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f398f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f39900cf7c0 0x7f399015ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39014/0 (socket says 172.21.15.33:39014) 2023-10-06T02:46:03.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f398f7fe700 1 -- 172.21.15.33:0/3490871827 learned_addr learned my addr 172.21.15.33:0/3490871827 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f398ffff700 1 -- 172.21.15.33:0/3490871827 >> v1:172.21.15.167:6789/0 conn(0x7f3990159030 legacy=0x7f399015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.497+0000 7f398ffff700 1 -- 172.21.15.33:0/3490871827 >> v1:172.21.15.33:6789/0 conn(0x7f39900cf7c0 legacy=0x7f399015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f398ffff700 1 -- 172.21.15.33:0/3490871827 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39900ce860 con 0x7f3990158c60 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f398ffff700 1 --2- 172.21.15.33:0/3490871827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990158c60 0x7f3990068390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f39800098d0 tx=0x7f3980009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59964d5bcf41373 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f398effd700 1 -- 172.21.15.33:0/3490871827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980009e10 con 0x7f3990158c60 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f398effd700 1 -- 172.21.15.33:0/3490871827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398000b760 con 0x7f3990158c60 2023-10-06T02:46:03.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f398effd700 1 -- 172.21.15.33:0/3490871827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398000bb00 con 0x7f3990158c60 2023-10-06T02:46:03.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 -- 172.21.15.33:0/3490871827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990158c60 msgr2=0x7f3990068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 --2- 172.21.15.33:0/3490871827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990158c60 0x7f3990068390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f39800098d0 tx=0x7f3980009120 comp rx=0 tx=0).stop 2023-10-06T02:46:03.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 -- 172.21.15.33:0/3490871827 shutdown_connections 2023-10-06T02:46:03.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 --2- 172.21.15.33:0/3490871827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990158c60 0x7f3990068390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 -- 172.21.15.33:0/3490871827 >> 172.21.15.33:0/3490871827 conn(0x7f3990067a40 msgr2=0x7f39900b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:03.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 -- 172.21.15.33:0/3490871827 shutdown_connections 2023-10-06T02:46:03.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.501+0000 7f3995308700 1 -- 172.21.15.33:0/3490871827 wait complete. 2023-10-06T02:46:03.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 Processor -- start 2023-10-06T02:46:03.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 -- start start 2023-10-06T02:46:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900cf7c0 0x7f39901518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3990158c60 0x7f3990157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 0x7f3990157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39901697b0 con 0x7f3990159030 2023-10-06T02:46:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399016dd70 con 0x7f39900cf7c0 2023-10-06T02:46:03.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3995308700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399016a000 con 0x7f3990158c60 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 0x7f3990157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 0x7f3990157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54046/0 (socket says 172.21.15.33:54046) 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 -- 172.21.15.33:0/2251137018 learned_addr learned my addr 172.21.15.33:0/2251137018 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f398f7fe700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3990158c60 0x7f3990157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f398ffff700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900cf7c0 0x7f39901518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 -- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3990158c60 msgr2=0x7f3990157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3990158c60 0x7f3990157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 -- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900cf7c0 msgr2=0x7f39901518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900cf7c0 0x7f39901518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.505+0000 7f3994b07700 1 -- 172.21.15.33:0/2251137018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800093c0 con 0x7f3990159030 2023-10-06T02:46:03.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3994b07700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 0x7f3990157a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f398800c870 tx=0x7f3988001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988012630 con 0x7f3990159030 2023-10-06T02:46:03.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880127b0 con 0x7f3990159030 2023-10-06T02:46:03.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399016da90 con 0x7f3990159030 2023-10-06T02:46:03.516 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988012b70 con 0x7f3990159030 2023-10-06T02:46:03.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3990169c90 con 0x7f3990159030 2023-10-06T02:46:03.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3990071040 con 0x7f3990159030 2023-10-06T02:46:03.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3988003030 con 0x7f3990159030 2023-10-06T02:46:03.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3977fff700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f397c060580 0x7f397c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.509+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f398807ca30 con 0x7f3990159030 2023-10-06T02:46:03.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.513+0000 7f398ffff700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f397c060580 0x7f397c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.513+0000 7f398ffff700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f397c060580 0x7f397c062a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39800030c0 tx=0x7f3980016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.517+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f398804cb90 con 0x7f3990159030 2023-10-06T02:46:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39900ceb40 con 0x7f3990159030 2023-10-06T02:46:03.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3977fff700 1 -- 172.21.15.33:0/2251137018 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39880501b0 con 0x7f3990159030 2023-10-06T02:46:03.686 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f397c060580 msgr2=0x7f397c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f397c060580 0x7f397c062a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39800030c0 tx=0x7f3980016000 comp rx=0 tx=0).stop 2023-10-06T02:46:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 msgr2=0x7f3990157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 0x7f3990157a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f398800c870 tx=0x7f3988001670 comp rx=0 tx=0).stop 2023-10-06T02:46:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 shutdown_connections 2023-10-06T02:46:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f397c060580 0x7f397c062a30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3990158c60 0x7f3990157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3990159030 0x7f3990157a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 --2- 172.21.15.33:0/2251137018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39900cf7c0 0x7f39901518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 >> 172.21.15.33:0/2251137018 conn(0x7f3990067a40 msgr2=0x7f3990068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.681+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 shutdown_connections 2023-10-06T02:46:03.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.685+0000 7f3995308700 1 -- 172.21.15.33:0/2251137018 wait complete. 2023-10-06T02:46:03.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 Processor -- start 2023-10-06T02:46:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 -- start start 2023-10-06T02:46:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0200b2690 0x7fd0200aa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd020155df0 0x7fd0200b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0200aa2f0 con 0x7fd020155a20 2023-10-06T02:46:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0200a9720 con 0x7fd020155df0 2023-10-06T02:46:03.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd027d23700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200a99c0 con 0x7fd0200b2690 2023-10-06T02:46:03.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd0262c0700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd020155df0 0x7fd0200b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd025abf700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd020155a20 0x7fd020068ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39020/0 (socket says 172.21.15.33:39020) 2023-10-06T02:46:03.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd0262c0700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd020155df0 0x7fd0200b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54062/0 (socket says 172.21.15.33:54062) 2023-10-06T02:46:03.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd025abf700 1 -- 172.21.15.33:0/2030250513 learned_addr learned my addr 172.21.15.33:0/2030250513 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:03.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.813+0000 7fd0252be700 1 --2- 172.21.15.33:0/2030250513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0200b2690 0x7fd0200aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd0262c0700 1 -- 172.21.15.33:0/2030250513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0200b2690 msgr2=0x7fd0200aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd0262c0700 1 --2- 172.21.15.33:0/2030250513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0200b2690 0x7fd0200aa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd0262c0700 1 -- 172.21.15.33:0/2030250513 >> v1:172.21.15.33:6789/0 conn(0x7fd020155a20 legacy=0x7fd020068ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd024abd700 1 -- 172.21.15.33:0/2030250513 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144630552 0 0) 0x7fd0200aa2f0 con 0x7fd020155a20 2023-10-06T02:46:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd0262c0700 1 -- 172.21.15.33:0/2030250513 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200a9ca0 con 0x7fd020155df0 2023-10-06T02:46:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd0262c0700 1 --2- 172.21.15.33:0/2030250513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd020155df0 0x7fd0200b1a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e60 tx=0x7fd018009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49eb0dbaa62dc88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd024abd700 1 -- 172.21.15.33:0/2030250513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01800e070 con 0x7fd020155df0 2023-10-06T02:46:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd024abd700 1 -- 172.21.15.33:0/2030250513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01800bb60 con 0x7fd020155df0 2023-10-06T02:46:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd024abd700 1 -- 172.21.15.33:0/2030250513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018012430 con 0x7fd020155df0 2023-10-06T02:46:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 -- 172.21.15.33:0/2030250513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd020155df0 msgr2=0x7fd0200b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 --2- 172.21.15.33:0/2030250513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd020155df0 0x7fd0200b1a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e60 tx=0x7fd018009780 comp rx=0 tx=0).stop 2023-10-06T02:46:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 -- 172.21.15.33:0/2030250513 shutdown_connections 2023-10-06T02:46:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 --2- 172.21.15.33:0/2030250513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0200b2690 0x7fd0200aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 --2- 172.21.15.33:0/2030250513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd020155df0 0x7fd0200b1a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 -- 172.21.15.33:0/2030250513 >> 172.21.15.33:0/2030250513 conn(0x7fd0200af9c0 msgr2=0x7fd0200afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 -- 172.21.15.33:0/2030250513 shutdown_connections 2023-10-06T02:46:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.817+0000 7fd027d23700 1 -- 172.21.15.33:0/2030250513 wait complete. 2023-10-06T02:46:03.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 Processor -- start 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 -- start start 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 0x7fd0201416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd020155a20 0x7fd020156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd020155df0 0x7fd020146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd020141c00 con 0x7fd0200b2690 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200ac4c0 con 0x7fd020155a20 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd027d23700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200ac7a0 con 0x7fd020155df0 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd025abf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 0x7fd0201416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd025abf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 0x7fd0201416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54074/0 (socket says 172.21.15.33:54074) 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd025abf700 1 -- 172.21.15.33:0/930981384 learned_addr learned my addr 172.21.15.33:0/930981384 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd0252be700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd020155a20 0x7fd020156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.821+0000 7fd0262c0700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd020155df0 0x7fd020146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd025abf700 1 -- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd020155df0 msgr2=0x7fd020146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd025abf700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd020155df0 0x7fd020146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd025abf700 1 -- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd020155a20 msgr2=0x7fd020156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd025abf700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd020155a20 0x7fd020156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd025abf700 1 -- 172.21.15.33:0/930981384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009a20 con 0x7fd0200b2690 2023-10-06T02:46:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd025abf700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 0x7fd0201416c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd01000c7e0 tx=0x7fd010007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010005b30 con 0x7fd0200b2690 2023-10-06T02:46:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd010005c90 con 0x7fd0200b2690 2023-10-06T02:46:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0200aca80 con 0x7fd0200b2690 2023-10-06T02:46:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010007300 con 0x7fd0200b2690 2023-10-06T02:46:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0200ad040 con 0x7fd0200b2690 2023-10-06T02:46:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0200ad260 con 0x7fd0200b2690 2023-10-06T02:46:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd01001b3b0 con 0x7fd0200b2690 2023-10-06T02:46:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd015ffb700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd004060550 0x7fd004062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.825+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fd01007c570 con 0x7fd0200b2690 2023-10-06T02:46:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.829+0000 7fd0252be700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd004060550 0x7fd004062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.829+0000 7fd0252be700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd004060550 0x7fd004062a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd02016a350 tx=0x7fd01c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.833+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd01004c750 con 0x7fd0200b2690 2023-10-06T02:46:04.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd0200710b0 con 0x7fd0200b2690 2023-10-06T02:46:04.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd015ffb700 1 -- 172.21.15.33:0/930981384 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0100170a0 con 0x7fd0200b2690 2023-10-06T02:46:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd004060550 msgr2=0x7fd004062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd004060550 0x7fd004062a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd02016a350 tx=0x7fd01c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 msgr2=0x7fd0201416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 0x7fd0201416c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd01000c7e0 tx=0x7fd010007df0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 shutdown_connections 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd004060550 0x7fd004062a00 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd02016a350 tx=0x7fd01c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd020155df0 0x7fd020146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd020155a20 0x7fd020156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 --2- 172.21.15.33:0/930981384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0200b2690 0x7fd0201416c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 >> 172.21.15.33:0/930981384 conn(0x7fd0200af9c0 msgr2=0x7fd02014b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 shutdown_connections 2023-10-06T02:46:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:03.997+0000 7fd027d23700 1 -- 172.21.15.33:0/930981384 wait complete. 2023-10-06T02:46:04.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:46:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a8ff2700 1 Processor -- start 2023-10-06T02:46:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a8ff2700 1 -- start start 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a8ff2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cde40 0x7f80a406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a8ff2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40cc640 con 0x7f80a4156500 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a8ff2700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40cc920 con 0x7f80a40cda70 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a8ff2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a406c6b0 con 0x7f80a40cde40 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f80a4156500 0x7f80a4156980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39030/0 (socket says 172.21.15.33:39030) 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a3fff700 1 -- 172.21.15.33:0/3050029826 learned_addr learned my addr 172.21.15.33:0/3050029826 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a2ffd700 1 --2- 172.21.15.33:0/3050029826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cde40 0x7f80a406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515425742 0 0) 0x7f80a40cc640 con 0x7f80a4156500 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f808c003570 con 0x7f80a4156500 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679582921 0 0) 0x7f80a40cc920 con 0x7f80a40cda70 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a40cc640 con 0x7f80a40cda70 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a2ffd700 1 -- 172.21.15.33:0/3050029826 >> v1:172.21.15.167:6789/0 conn(0x7f80a40cda70 legacy=0x7f80a40cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a2ffd700 1 -- 172.21.15.33:0/3050029826 >> v1:172.21.15.33:6789/0 conn(0x7f80a4156500 legacy=0x7f80a4156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.129+0000 7f80a2ffd700 1 -- 172.21.15.33:0/3050029826 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40ccc00 con 0x7f80a40cde40 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a2ffd700 1 --2- 172.21.15.33:0/3050029826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cde40 0x7f80a406c170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8094009910 tx=0x7f80940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3479c9eb7f4aa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094011630 con 0x7f80a40cde40 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8094011790 con 0x7f80a40cde40 2023-10-06T02:46:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a27fc700 1 -- 172.21.15.33:0/3050029826 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094011b30 con 0x7f80a40cde40 2023-10-06T02:46:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 -- 172.21.15.33:0/3050029826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cde40 msgr2=0x7f80a406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/3050029826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cde40 0x7f80a406c170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8094009910 tx=0x7f80940091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 -- 172.21.15.33:0/3050029826 shutdown_connections 2023-10-06T02:46:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/3050029826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cde40 0x7f80a406c170 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 -- 172.21.15.33:0/3050029826 >> 172.21.15.33:0/3050029826 conn(0x7f80a406ad60 msgr2=0x7f80a406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 -- 172.21.15.33:0/3050029826 shutdown_connections 2023-10-06T02:46:04.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 -- 172.21.15.33:0/3050029826 wait complete. 2023-10-06T02:46:04.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 Processor -- start 2023-10-06T02:46:04.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.133+0000 7f80a8ff2700 1 -- start start 2023-10-06T02:46:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cda70 0x7f80a415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80a40cde40 0x7f80a4152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 0x7f80a40c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a415ab30 con 0x7f80a4156500 2023-10-06T02:46:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a416b9e0 con 0x7f80a40cde40 2023-10-06T02:46:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a416add0 con 0x7f80a40cda70 2023-10-06T02:46:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 0x7f80a40c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a37fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cda70 0x7f80a415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 0x7f80a40c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54080/0 (socket says 172.21.15.33:54080) 2023-10-06T02:46:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 -- 172.21.15.33:0/2292508786 learned_addr learned my addr 172.21.15.33:0/2292508786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a2ffd700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80a40cde40 0x7f80a4152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 -- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cda70 msgr2=0x7f80a415a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cda70 0x7f80a415a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 -- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80a40cde40 msgr2=0x7f80a4152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80a40cde40 0x7f80a4152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 -- 172.21.15.33:0/2292508786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009420 con 0x7f80a4156500 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a3fff700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 0x7f80a40c7f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f809c00cd00 tx=0x7f809c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c00ce00 con 0x7f80a4156500 2023-10-06T02:46:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c00aac0 con 0x7f80a4156500 2023-10-06T02:46:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a416b0b0 con 0x7f80a4156500 2023-10-06T02:46:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c00ae80 con 0x7f80a4156500 2023-10-06T02:46:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a41539b0 con 0x7f80a4156500 2023-10-06T02:46:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.137+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4071040 con 0x7f80a4156500 2023-10-06T02:46:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.141+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f809c007300 con 0x7f80a4156500 2023-10-06T02:46:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.141+0000 7f808b7fe700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8090060550 0x7f8090062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.141+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f809c07d010 con 0x7f80a4156500 2023-10-06T02:46:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.141+0000 7f80a37fe700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8090060550 0x7f8090062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.141+0000 7f80a37fe700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8090060550 0x7f8090062a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80a40690f0 tx=0x7f8098007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.149+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f809c04d170 con 0x7f80a4156500 2023-10-06T02:46:04.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.309+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80a406d3e0 con 0x7f80a4156500 2023-10-06T02:46:04.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.309+0000 7f808b7fe700 1 -- 172.21.15.33:0/2292508786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f809c050790 con 0x7f80a4156500 2023-10-06T02:46:04.315 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8090060550 msgr2=0x7f8090062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8090060550 0x7f8090062a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80a40690f0 tx=0x7f8098007d50 comp rx=0 tx=0).stop 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 msgr2=0x7f80a40c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 0x7f80a40c7f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f809c00cd00 tx=0x7f809c001ac0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 shutdown_connections 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a4156500 0x7f80a40c7f60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80a40cde40 0x7f80a4152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8090060550 0x7f8090062a00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 --2- 172.21.15.33:0/2292508786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80a40cda70 0x7f80a415a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 >> 172.21.15.33:0/2292508786 conn(0x7f80a406ad60 msgr2=0x7f80a4155550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 shutdown_connections 2023-10-06T02:46:04.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.313+0000 7f80a8ff2700 1 -- 172.21.15.33:0/2292508786 wait complete. 2023-10-06T02:46:04.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:46:04.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 Processor -- start 2023-10-06T02:46:04.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 -- start start 2023-10-06T02:46:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2bc80cc000 0x7f2bc806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc8067b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc8068ed0 con 0x7f2bc80cc3d0 2023-10-06T02:46:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc81591f0 con 0x7f2bc80cc000 2023-10-06T02:46:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcef48700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8068070 con 0x7f2bc80c7840 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bce747700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc8067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcdf46700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2bc80cc000 0x7f2bc806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bce747700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc8067b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40000/0 (socket says 172.21.15.33:40000) 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bce747700 1 -- 172.21.15.33:0/1395405975 learned_addr learned my addr 172.21.15.33:0/1395405975 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.437+0000 7f2bcd745700 1 --1- 172.21.15.33:0/1395405975 >> v1:172.21.15.167:6789/0 conn(0x7f2bc80cc3d0 0x7f2bc80c7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47132/0 (socket says 172.21.15.33:47132) 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bccf44700 1 -- 172.21.15.33:0/1395405975 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671858060 0 0) 0x7f2bc8068ed0 con 0x7f2bc80cc3d0 2023-10-06T02:46:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bce747700 1 -- 172.21.15.33:0/1395405975 >> v2:172.21.15.167:3300/0 conn(0x7f2bc80cc000 msgr2=0x7f2bc806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bce747700 1 --2- 172.21.15.33:0/1395405975 >> v2:172.21.15.167:3300/0 conn(0x7f2bc80cc000 0x7f2bc806c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bce747700 1 -- 172.21.15.33:0/1395405975 >> v1:172.21.15.167:6789/0 conn(0x7f2bc80cc3d0 legacy=0x7f2bc80c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bce747700 1 -- 172.21.15.33:0/1395405975 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80682c0 con 0x7f2bc80c7840 2023-10-06T02:46:04.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bce747700 1 --2- 172.21.15.33:0/1395405975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc8067b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009f20 tx=0x7f2bc0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f47acb6783f853 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bccf44700 1 -- 172.21.15.33:0/1395405975 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000e070 con 0x7f2bc80c7840 2023-10-06T02:46:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bccf44700 1 -- 172.21.15.33:0/1395405975 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000b9d0 con 0x7f2bc80c7840 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bccf44700 1 -- 172.21.15.33:0/1395405975 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00092e0 con 0x7f2bc80c7840 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 -- 172.21.15.33:0/1395405975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 msgr2=0x7f2bc8067b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 --2- 172.21.15.33:0/1395405975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc8067b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009f20 tx=0x7f2bc0009840 comp rx=0 tx=0).stop 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 -- 172.21.15.33:0/1395405975 shutdown_connections 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 --2- 172.21.15.33:0/1395405975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc8067b30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 --2- 172.21.15.33:0/1395405975 >> v2:172.21.15.167:3300/0 conn(0x7f2bc80cc000 0x7f2bc806c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 -- 172.21.15.33:0/1395405975 >> 172.21.15.33:0/1395405975 conn(0x7f2bc80c9350 msgr2=0x7f2bc80c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 -- 172.21.15.33:0/1395405975 shutdown_connections 2023-10-06T02:46:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.441+0000 7f2bcef48700 1 -- 172.21.15.33:0/1395405975 wait complete. 2023-10-06T02:46:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 Processor -- start 2023-10-06T02:46:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 -- start start 2023-10-06T02:46:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc80b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bc80cc000 0x7f2bc80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 0x7f2bc8168920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8159960 con 0x7f2bc80cc3d0 2023-10-06T02:46:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8159330 con 0x7f2bc80cc000 2023-10-06T02:46:04.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bcef48700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8169490 con 0x7f2bc80c7840 2023-10-06T02:46:04.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.445+0000 7f2bce747700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 0x7f2bc8168920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bce747700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 0x7f2bc8168920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54090/0 (socket says 172.21.15.33:54090) 2023-10-06T02:46:04.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bc80cc000 0x7f2bc80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcdf46700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc80b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bce747700 1 -- 172.21.15.33:0/3943675987 learned_addr learned my addr 172.21.15.33:0/3943675987 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:04.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 -- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 msgr2=0x7f2bc80b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc80b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 -- 172.21.15.33:0/3943675987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 msgr2=0x7f2bc8168920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 0x7f2bc8168920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 -- 172.21.15.33:0/3943675987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009ae0 con 0x7f2bc80cc000 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bce747700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 0x7f2bc8168920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcd745700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bc80cc000 0x7f2bc80b0650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c4f0 tx=0x7f2bb8007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800cea0 con 0x7f2bc80cc000 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8005a70 con 0x7f2bc80cc000 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc8169770 con 0x7f2bc80cc000 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8005e30 con 0x7f2bc80cc000 2023-10-06T02:46:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc80b24c0 con 0x7f2bc80cc000 2023-10-06T02:46:04.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8159aa0 con 0x7f2bc80cc000 2023-10-06T02:46:04.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.449+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2bb80072e0 con 0x7f2bc80cc000 2023-10-06T02:46:04.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.453+0000 7f2bbdffb700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2bac060550 0x7f2bac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.453+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f2bb807c070 con 0x7f2bc80cc000 2023-10-06T02:46:04.457 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.453+0000 7f2bcdf46700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2bac060550 0x7f2bac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.453+0000 7f2bcdf46700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2bac060550 0x7f2bac062a00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009da0 tx=0x7f2bc4009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.461+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2bb80abbe0 con 0x7f2bc80cc000 2023-10-06T02:46:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.625+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bc80b1be0 con 0x7f2bc80cc000 2023-10-06T02:46:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.625+0000 7f2bbdffb700 1 -- 172.21.15.33:0/3943675987 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bb80170a0 con 0x7f2bc80cc000 2023-10-06T02:46:04.632 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:46:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2bac060550 msgr2=0x7f2bac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2bac060550 0x7f2bac062a00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009da0 tx=0x7f2bc4009120 comp rx=0 tx=0).stop 2023-10-06T02:46:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bc80cc000 msgr2=0x7f2bc80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bc80cc000 0x7f2bc80b0650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c4f0 tx=0x7f2bb8007ef0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 shutdown_connections 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bc80cc3d0 0x7f2bc8168920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bc80cc000 0x7f2bc80b0650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2bac060550 0x7f2bac062a00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 --2- 172.21.15.33:0/3943675987 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2bc80c7840 0x7f2bc80b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 >> 172.21.15.33:0/3943675987 conn(0x7f2bc80c9350 msgr2=0x7f2bc80cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 shutdown_connections 2023-10-06T02:46:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.629+0000 7f2bcef48700 1 -- 172.21.15.33:0/3943675987 wait complete. 2023-10-06T02:46:04.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:46:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.757+0000 7f6192533700 1 Processor -- start 2023-10-06T02:46:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.757+0000 7f6192533700 1 -- start start 2023-10-06T02:46:04.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f618c159f80 0x7f618c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f618c0cb6e0 0x7f618c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c15a400 con 0x7f618c1557a0 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0cbb40 con 0x7f618c0cb6e0 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c159af0 con 0x7f618c159f80 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191d32700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f618c0cb6e0 0x7f618c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191531700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f618c1557a0 0x7f618c0c79c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39038/0 (socket says 172.21.15.33:39038) 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191531700 1 -- 172.21.15.33:0/1529086379 learned_addr learned my addr 172.21.15.33:0/1529086379 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6190d30700 1 --2- 172.21.15.33:0/1529086379 >> v2:172.21.15.167:3300/0 conn(0x7f618c159f80 0x7f618c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191d32700 1 -- 172.21.15.33:0/1529086379 >> v2:172.21.15.167:3300/0 conn(0x7f618c159f80 msgr2=0x7f618c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191d32700 1 --2- 172.21.15.33:0/1529086379 >> v2:172.21.15.167:3300/0 conn(0x7f618c159f80 0x7f618c0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191d32700 1 -- 172.21.15.33:0/1529086379 >> v1:172.21.15.33:6789/0 conn(0x7f618c1557a0 legacy=0x7f618c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191d32700 1 -- 172.21.15.33:0/1529086379 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c1594e0 con 0x7f618c0cb6e0 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6191d32700 1 --2- 172.21.15.33:0/1529086379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c0cb6e0 0x7f618c155b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f618c06d970 tx=0x7f6184009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d950608ef7736 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f617ffff700 1 -- 172.21.15.33:0/1529086379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618400e070 con 0x7f618c0cb6e0 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f617ffff700 1 -- 172.21.15.33:0/1529086379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618400bb60 con 0x7f618c0cb6e0 2023-10-06T02:46:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f617ffff700 1 -- 172.21.15.33:0/1529086379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184012430 con 0x7f618c0cb6e0 2023-10-06T02:46:04.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- 172.21.15.33:0/1529086379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c0cb6e0 msgr2=0x7f618c155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 --2- 172.21.15.33:0/1529086379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c0cb6e0 0x7f618c155b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f618c06d970 tx=0x7f6184009780 comp rx=0 tx=0).stop 2023-10-06T02:46:04.775 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- 172.21.15.33:0/1529086379 shutdown_connections 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 --2- 172.21.15.33:0/1529086379 >> v2:172.21.15.167:3300/0 conn(0x7f618c159f80 0x7f618c0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 --2- 172.21.15.33:0/1529086379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c0cb6e0 0x7f618c155b70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- 172.21.15.33:0/1529086379 >> 172.21.15.33:0/1529086379 conn(0x7f618c0c8ad0 msgr2=0x7f618c0c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- 172.21.15.33:0/1529086379 shutdown_connections 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.761+0000 7f6192533700 1 -- 172.21.15.33:0/1529086379 wait complete. 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 Processor -- start 2023-10-06T02:46:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 -- start start 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c0cb6e0 0x7f618c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 0x7f618c0ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c159f80 0x7f618c14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0b09c0 con 0x7f618c1557a0 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c153330 con 0x7f618c159f80 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6192533700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c153610 con 0x7f618c0cb6e0 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6190d30700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 0x7f618c0ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.777 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6190d30700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 0x7f618c0ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54112/0 (socket says 172.21.15.33:54112) 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6190d30700 1 -- 172.21.15.33:0/1673092570 learned_addr learned my addr 172.21.15.33:0/1673092570 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.765+0000 7f6191d32700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c159f80 0x7f618c14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6190d30700 1 -- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c0cb6e0 msgr2=0x7f618c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6191531700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c0cb6e0 0x7f618c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6190d30700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c0cb6e0 0x7f618c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6190d30700 1 -- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c159f80 msgr2=0x7f618c14ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6190d30700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c159f80 0x7f618c14ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6190d30700 1 -- 172.21.15.33:0/1673092570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009a20 con 0x7f618c1557a0 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6191d32700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c159f80 0x7f618c14ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6191531700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c0cb6e0 0x7f618c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6190d30700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 0x7f618c0ad290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f618800ec80 tx=0x7f61880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800c9d0 con 0x7f618c1557a0 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618800cb30 con 0x7f618c1557a0 2023-10-06T02:46:04.779 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c1538f0 con 0x7f618c1557a0 2023-10-06T02:46:04.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188013470 con 0x7f618c1557a0 2023-10-06T02:46:04.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c0b05e0 con 0x7f618c1557a0 2023-10-06T02:46:04.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c0b0b30 con 0x7f618c1557a0 2023-10-06T02:46:04.780 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.769+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6188013820 con 0x7f618c1557a0 2023-10-06T02:46:04.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.773+0000 7f617d7fa700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6170060550 0x7f6170062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:04.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.773+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f618807cfa0 con 0x7f618c1557a0 2023-10-06T02:46:04.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.773+0000 7f6191531700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6170060550 0x7f6170062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:04.781 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.773+0000 7f6191531700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6170060550 0x7f6170062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6180001820 tx=0x7f6180009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:04.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.781+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f61880acbe0 con 0x7f618c1557a0 2023-10-06T02:46:04.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f618c071040 con 0x7f618c1557a0 2023-10-06T02:46:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f617d7fa700 1 -- 172.21.15.33:0/1673092570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v31) v1 ==== 103+0+11 (secure 0 0 0) 0x7f618804d100 con 0x7f618c1557a0 2023-10-06T02:46:04.949 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6170060550 msgr2=0x7f6170062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6170060550 0x7f6170062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6180001820 tx=0x7f6180009040 comp rx=0 tx=0).stop 2023-10-06T02:46:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 msgr2=0x7f618c0ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 0x7f618c0ad290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f618800ec80 tx=0x7f61880091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 shutdown_connections 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f618c159f80 0x7f618c14ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c1557a0 0x7f618c0ad290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6170060550 0x7f6170062a00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 --2- 172.21.15.33:0/1673092570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f618c0cb6e0 0x7f618c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 >> 172.21.15.33:0/1673092570 conn(0x7f618c0c8ad0 msgr2=0x7f618c0ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 shutdown_connections 2023-10-06T02:46:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:04.945+0000 7f6192533700 1 -- 172.21.15.33:0/1673092570 wait complete. 2023-10-06T02:46:04.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 Processor -- start 2023-10-06T02:46:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 -- start start 2023-10-06T02:46:05.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 0x7f6ca8159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca80b3860 con 0x7f6ca80af930 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca815d1f0 con 0x7f6ca8153770 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cae3ca700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca815cbe0 con 0x7f6ca80af560 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cac967700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 0x7f6ca8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cac967700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 0x7f6ca8159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54120/0 (socket says 172.21.15.33:54120) 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cac967700 1 -- 172.21.15.33:0/99739564 learned_addr learned my addr 172.21.15.33:0/99739564 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:05.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/99739564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/99739564 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602721262 0 0) 0x7f6ca80b3860 con 0x7f6ca80af930 2023-10-06T02:46:05.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca7fff700 1 -- 172.21.15.33:0/99739564 >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 msgr2=0x7f6ca8159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:05.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/99739564 >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 0x7f6ca8159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca7fff700 1 -- 172.21.15.33:0/99739564 >> v1:172.21.15.167:6789/0 conn(0x7f6ca80af930 legacy=0x7f6ca8153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca7fff700 1 -- 172.21.15.33:0/99739564 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca815a190 con 0x7f6ca80af560 2023-10-06T02:46:05.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6cac967700 1 --2- 172.21.15.33:0/99739564 >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 0x7f6ca8159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/99739564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca815d680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009e60 tx=0x7f6c98009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6584eb905d30a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.069+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/99739564 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800e070 con 0x7f6ca80af560 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/99739564 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9800bb60 con 0x7f6ca80af560 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6ca6ffd700 1 -- 172.21.15.33:0/99739564 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98012430 con 0x7f6ca80af560 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- 172.21.15.33:0/99739564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 msgr2=0x7f6ca815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/99739564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca815d680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009e60 tx=0x7f6c98009780 comp rx=0 tx=0).stop 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- 172.21.15.33:0/99739564 shutdown_connections 2023-10-06T02:46:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/99739564 >> v2:172.21.15.33:3300/0 conn(0x7f6ca8153770 0x7f6ca8159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/99739564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca815d680 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- 172.21.15.33:0/99739564 >> 172.21.15.33:0/99739564 conn(0x7f6ca80cdb90 msgr2=0x7f6ca80cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:05.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- 172.21.15.33:0/99739564 shutdown_connections 2023-10-06T02:46:05.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- 172.21.15.33:0/99739564 wait complete. 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 Processor -- start 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- start start 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca8155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca80af930 0x7f6ca81692d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 0x7f6ca8169810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81574f0 con 0x7f6ca8153770 2023-10-06T02:46:05.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8156450 con 0x7f6ca80af930 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.073+0000 7f6cae3ca700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8156730 con 0x7f6ca80af560 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 0x7f6ca8169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 0x7f6ca8169810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54134/0 (socket says 172.21.15.33:54134) 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 -- 172.21.15.33:0/1104306795 learned_addr learned my addr 172.21.15.33:0/1104306795 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca8155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6ca77fe700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca80af930 0x7f6ca81692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 -- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 msgr2=0x7f6ca8155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca8155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 -- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca80af930 msgr2=0x7f6ca81692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca80af930 0x7f6ca81692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 -- 172.21.15.33:0/1104306795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00094f0 con 0x7f6ca8153770 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6ca77fe700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca80af930 0x7f6ca81692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca8155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cac967700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 0x7f6ca8169810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ea40 tx=0x7f6ca0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000c850 con 0x7f6ca8153770 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca000c9b0 con 0x7f6ca8153770 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c98009a20 con 0x7f6ca8153770 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000cd50 con 0x7f6ca8153770 2023-10-06T02:46:05.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca8156cf0 con 0x7f6ca8153770 2023-10-06T02:46:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca815a560 con 0x7f6ca8153770 2023-10-06T02:46:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6ca0013680 con 0x7f6ca8153770 2023-10-06T02:46:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6c9ffff700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6c88060550 0x7f6c88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f6ca007cf30 con 0x7f6ca8153770 2023-10-06T02:46:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.077+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6c88060550 0x7f6c88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.085 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.081+0000 7f6ca7fff700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6c88060550 0x7f6c88062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c980097c0 tx=0x7f6c98019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.085+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6ca815a560 con 0x7f6ca8153770 2023-10-06T02:46:05.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.249+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ca815a560 con 0x7f6ca8153770 2023-10-06T02:46:05.254 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.249+0000 7f6c9ffff700 1 -- 172.21.15.33:0/1104306795 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f6ca815a560 con 0x7f6ca8153770 2023-10-06T02:46:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:05.255 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:04.343265+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:05.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6c88060550 msgr2=0x7f6c88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6c88060550 0x7f6c88062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c980097c0 tx=0x7f6c98019000 comp rx=0 tx=0).stop 2023-10-06T02:46:05.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 msgr2=0x7f6ca8169810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 0x7f6ca8169810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ea40 tx=0x7f6ca0009250 comp rx=0 tx=0).stop 2023-10-06T02:46:05.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 shutdown_connections 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca8153770 0x7f6ca8169810 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ca80af930 0x7f6ca81692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6c88060550 0x7f6c88062a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 --2- 172.21.15.33:0/1104306795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ca80af560 0x7f6ca8155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 >> 172.21.15.33:0/1104306795 conn(0x7f6ca80cdb90 msgr2=0x7f6ca8152980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 shutdown_connections 2023-10-06T02:46:05.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.253+0000 7f6cae3ca700 1 -- 172.21.15.33:0/1104306795 wait complete. 2023-10-06T02:46:05.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:46:05.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.369+0000 7f60b8757700 1 Processor -- start 2023-10-06T02:46:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.369+0000 7f60b8757700 1 -- start start 2023-10-06T02:46:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60b0158cf0 0x7f60b00aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b01590c0 0x7f60b00af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60b0068e40 0x7f60b00692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b006a160 con 0x7f60b0158cf0 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b00cda90 con 0x7f60b0068e40 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00b1720 con 0x7f60b01590c0 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60b0158cf0 0x7f60b00aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f60b0158cf0 0x7f60b00aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54140/0 (socket says 172.21.15.33:54140) 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 -- 172.21.15.33:0/3740107711 learned_addr learned my addr 172.21.15.33:0/3740107711 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b6cf4700 1 --2- 172.21.15.33:0/3740107711 >> v2:172.21.15.167:3300/0 conn(0x7f60b0068e40 0x7f60b00692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b5cf2700 1 --2- 172.21.15.33:0/3740107711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b01590c0 0x7f60b00af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 -- 172.21.15.33:0/3740107711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b01590c0 msgr2=0x7f60b00af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 --2- 172.21.15.33:0/3740107711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b01590c0 0x7f60b00af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 -- 172.21.15.33:0/3740107711 >> v2:172.21.15.167:3300/0 conn(0x7f60b0068e40 msgr2=0x7f60b00692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 --2- 172.21.15.33:0/3740107711 >> v2:172.21.15.167:3300/0 conn(0x7f60b0068e40 0x7f60b00692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 -- 172.21.15.33:0/3740107711 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00b1110 con 0x7f60b0158cf0 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b6cf4700 1 --2- 172.21.15.33:0/3740107711 >> v2:172.21.15.167:3300/0 conn(0x7f60b0068e40 0x7f60b00692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b64f3700 1 --2- 172.21.15.33:0/3740107711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b00aebe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60b00b0f00 tx=0x7f60a0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac42da6e0157aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b54f1700 1 -- 172.21.15.33:0/3740107711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0005c10 con 0x7f60b0158cf0 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b54f1700 1 -- 172.21.15.33:0/3740107711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a0005d70 con 0x7f60b0158cf0 2023-10-06T02:46:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b54f1700 1 -- 172.21.15.33:0/3740107711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a000b840 con 0x7f60b0158cf0 2023-10-06T02:46:05.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- 172.21.15.33:0/3740107711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 msgr2=0x7f60b00aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- 172.21.15.33:0/3740107711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b00aebe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60b00b0f00 tx=0x7f60a0009220 comp rx=0 tx=0).stop 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- 172.21.15.33:0/3740107711 shutdown_connections 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- 172.21.15.33:0/3740107711 >> v2:172.21.15.167:3300/0 conn(0x7f60b0068e40 0x7f60b00692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- 172.21.15.33:0/3740107711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b01590c0 0x7f60b00af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 --2- 172.21.15.33:0/3740107711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b00aebe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- 172.21.15.33:0/3740107711 >> 172.21.15.33:0/3740107711 conn(0x7f60b0096e40 msgr2=0x7f60b006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- 172.21.15.33:0/3740107711 shutdown_connections 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.373+0000 7f60b8757700 1 -- 172.21.15.33:0/3740107711 wait complete. 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 Processor -- start 2023-10-06T02:46:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 -- start start 2023-10-06T02:46:05.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b0068e40 0x7f60b0150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.386 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b01590c0 0x7f60b015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b01512e0 con 0x7f60b0158cf0 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0169860 con 0x7f60b01590c0 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0169b40 con 0x7f60b0068e40 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b015b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54148/0 (socket says 172.21.15.33:54148) 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 -- 172.21.15.33:0/4181192223 learned_addr learned my addr 172.21.15.33:0/4181192223 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b64f3700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b0068e40 0x7f60b0150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b6cf4700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b01590c0 0x7f60b015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 -- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b0068e40 msgr2=0x7f60b0150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.387 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b0068e40 0x7f60b0150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 -- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b01590c0 msgr2=0x7f60b015bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b01590c0 0x7f60b015bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 -- 172.21.15.33:0/4181192223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0009480 con 0x7f60b0158cf0 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b5cf2700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b015b620 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f60a400d860 tx=0x7f60a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40125f0 con 0x7f60b0158cf0 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a4012770 con 0x7f60b0158cf0 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b0169e20 con 0x7f60b0158cf0 2023-10-06T02:46:05.388 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4012b30 con 0x7f60b0158cf0 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.377+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b01693f0 con 0x7f60b0158cf0 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.381+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f60a4013480 con 0x7f60b0158cf0 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.381+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b016a2a0 con 0x7f60b0158cf0 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.381+0000 7f60ae7fc700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f609c060550 0x7f609c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.381+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f60a407cfe0 con 0x7f60b0158cf0 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.381+0000 7f60b64f3700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f609c060550 0x7f609c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.381+0000 7f60b64f3700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f609c060550 0x7f609c062a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009260 tx=0x7f60a0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.389+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f60a404d100 con 0x7f60b0158cf0 2023-10-06T02:46:05.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.553+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60b00710b0 con 0x7f60b0158cf0 2023-10-06T02:46:05.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60ae7fc700 1 -- 172.21.15.33:0/4181192223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60a4050720 con 0x7f60b0158cf0 2023-10-06T02:46:05.560 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f609c060550 msgr2=0x7f609c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f609c060550 0x7f609c062a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009260 tx=0x7f60a0016000 comp rx=0 tx=0).stop 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 msgr2=0x7f60b015b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b015b620 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f60a400d860 tx=0x7f60a4008040 comp rx=0 tx=0).stop 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 shutdown_connections 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60b01590c0 0x7f60b015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60b0158cf0 0x7f60b015b620 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f609c060550 0x7f609c062a00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 --2- 172.21.15.33:0/4181192223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60b0068e40 0x7f60b0150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 >> 172.21.15.33:0/4181192223 conn(0x7f60b0096e40 msgr2=0x7f60b006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 shutdown_connections 2023-10-06T02:46:05.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.557+0000 7f60b8757700 1 -- 172.21.15.33:0/4181192223 wait complete. 2023-10-06T02:46:05.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.681+0000 7fbe87af6700 1 Processor -- start 2023-10-06T02:46:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.681+0000 7fbe87af6700 1 -- start start 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe80158bf0 0x7fbe800b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe800b0530 0x7fbe800b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe800b2610 con 0x7fbe80158fc0 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe80067c10 con 0x7fbe80158bf0 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8015d830 con 0x7fbe800b0530 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe86093700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbe80158fc0 0x7fbe8015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39046/0 (socket says 172.21.15.33:39046) 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe86093700 1 -- 172.21.15.33:0/4087646951 learned_addr learned my addr 172.21.15.33:0/4087646951 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85892700 1 --2- 172.21.15.33:0/4087646951 >> v2:172.21.15.167:3300/0 conn(0x7fbe80158bf0 0x7fbe800b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85091700 1 --2- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe800b0530 0x7fbe800b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe84890700 1 -- 172.21.15.33:0/4087646951 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591493375 0 0) 0x7fbe7c001110 con 0x7fbe80158fc0 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe84890700 1 -- 172.21.15.33:0/4087646951 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe6c003570 con 0x7fbe80158fc0 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85892700 1 -- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe800b0530 msgr2=0x7fbe800b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85892700 1 --2- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe800b0530 0x7fbe800b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85892700 1 -- 172.21.15.33:0/4087646951 >> v1:172.21.15.33:6789/0 conn(0x7fbe80158fc0 legacy=0x7fbe8015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85892700 1 -- 172.21.15.33:0/4087646951 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800b00a0 con 0x7fbe80158bf0 2023-10-06T02:46:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe85892700 1 --2- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158bf0 0x7fbe800b1b90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009e60 tx=0x7fbe78009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e64b675ba7b3714f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe84890700 1 -- 172.21.15.33:0/4087646951 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800e070 con 0x7fbe80158bf0 2023-10-06T02:46:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe84890700 1 -- 172.21.15.33:0/4087646951 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7800bab0 con 0x7fbe80158bf0 2023-10-06T02:46:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe84890700 1 -- 172.21.15.33:0/4087646951 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800be30 con 0x7fbe80158bf0 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158bf0 msgr2=0x7fbe800b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 --2- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158bf0 0x7fbe800b1b90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009e60 tx=0x7fbe78009780 comp rx=0 tx=0).stop 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- 172.21.15.33:0/4087646951 shutdown_connections 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 --2- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe800b0530 0x7fbe800b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 --2- 172.21.15.33:0/4087646951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158bf0 0x7fbe800b1b90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- 172.21.15.33:0/4087646951 >> 172.21.15.33:0/4087646951 conn(0x7fbe800cfd70 msgr2=0x7fbe800d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- 172.21.15.33:0/4087646951 shutdown_connections 2023-10-06T02:46:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.685+0000 7fbe87af6700 1 -- 172.21.15.33:0/4087646951 wait complete. 2023-10-06T02:46:05.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 Processor -- start 2023-10-06T02:46:05.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 -- start start 2023-10-06T02:46:05.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe80158bf0 0x7fbe80150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158fc0 0x7fbe800ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 0x7fbe8015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8016cc40 con 0x7fbe800ce770 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8016cf20 con 0x7fbe80158fc0 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe87af6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8016d200 con 0x7fbe80158bf0 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 0x7fbe8015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 0x7fbe8015ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54164/0 (socket says 172.21.15.33:54164) 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 -- 172.21.15.33:0/3736851409 learned_addr learned my addr 172.21.15.33:0/3736851409 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe85091700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158fc0 0x7fbe800ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe85892700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe80158bf0 0x7fbe80150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 -- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe80158bf0 msgr2=0x7fbe80150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe80158bf0 0x7fbe80150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 -- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158fc0 msgr2=0x7fbe800ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158fc0 0x7fbe800ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 -- 172.21.15.33:0/3736851409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009a20 con 0x7fbe800ce770 2023-10-06T02:46:05.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.689+0000 7fbe86093700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 0x7fbe8015ab90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c790 tx=0x7fbe7c007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c005b30 con 0x7fbe800ce770 2023-10-06T02:46:05.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c005c90 con 0x7fbe800ce770 2023-10-06T02:46:05.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8016d4e0 con 0x7fbe800ce770 2023-10-06T02:46:05.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0072e0 con 0x7fbe800ce770 2023-10-06T02:46:05.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe801507a0 con 0x7fbe800ce770 2023-10-06T02:46:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe800672b0 con 0x7fbe800ce770 2023-10-06T02:46:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbe7c01b3b0 con 0x7fbe800ce770 2023-10-06T02:46:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe71ffb700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbe64060550 0x7fbe64062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fbe7c07beb0 con 0x7fbe800ce770 2023-10-06T02:46:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.693+0000 7fbe85892700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbe64060550 0x7fbe64062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:05.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.697+0000 7fbe85892700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbe64060550 0x7fbe64062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe780097c0 tx=0x7fbe78019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:05.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.701+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbe800672b0 con 0x7fbe800ce770 2023-10-06T02:46:05.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe800b2610 con 0x7fbe800ce770 2023-10-06T02:46:05.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe71ffb700 1 -- 172.21.15.33:0/3736851409 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe800b2610 con 0x7fbe800ce770 2023-10-06T02:46:05.869 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbe64060550 msgr2=0x7fbe64062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbe64060550 0x7fbe64062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe780097c0 tx=0x7fbe78019000 comp rx=0 tx=0).stop 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 msgr2=0x7fbe8015ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 0x7fbe8015ab90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c790 tx=0x7fbe7c007c30 comp rx=0 tx=0).stop 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 shutdown_connections 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe800ce770 0x7fbe8015ab90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe80158fc0 0x7fbe800ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbe64060550 0x7fbe64062a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 --2- 172.21.15.33:0/3736851409 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe80158bf0 0x7fbe80150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 >> 172.21.15.33:0/3736851409 conn(0x7fbe800cfd70 msgr2=0x7fbe800cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 shutdown_connections 2023-10-06T02:46:05.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.865+0000 7fbe87af6700 1 -- 172.21.15.33:0/3736851409 wait complete. 2023-10-06T02:46:05.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:46:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 Processor -- start 2023-10-06T02:46:05.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 -- start start 2023-10-06T02:46:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5fe4158c60 0x7f5fe4159030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fe415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe40cd7f0 con 0x7f5fe40caae0 2023-10-06T02:46:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe40cddb0 con 0x7f5fe4158c60 2023-10-06T02:46:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe9536700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415d630 con 0x7f5fe415cb70 2023-10-06T02:46:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe8d35700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5fe40caae0 0x7f5fe40caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39058/0 (socket says 172.21.15.33:39058) 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe8d35700 1 -- 172.21.15.33:0/3179006930 learned_addr learned my addr 172.21.15.33:0/3179006930 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe37fe700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fe415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.993+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/3179006930 >> v2:172.21.15.167:3300/0 conn(0x7f5fe4158c60 0x7f5fe4159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922996915 0 0) 0x7f5fe40cd7f0 con 0x7f5fe40caae0 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd0003570 con 0x7f5fe40caae0 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3796602472 0 0) 0x7f5fd0003570 con 0x7f5fe40caae0 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 msgr2=0x7f5fe415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fe415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 >> v2:172.21.15.167:3300/0 conn(0x7f5fe4158c60 msgr2=0x7f5fe4159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 --2- 172.21.15.33:0/3179006930 >> v2:172.21.15.167:3300/0 conn(0x7f5fe4158c60 0x7f5fe4159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe415d910 con 0x7f5fe40caae0 2023-10-06T02:46:06.003 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f5fdc0017f0 con 0x7f5fe40caae0 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 >> v1:172.21.15.33:6789/0 conn(0x7f5fe40caae0 legacy=0x7f5fe40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd0003a60 0x7f5fd0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fd0004810 0x7f5fd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe40cddb0 con 0x7f5fd0003a60 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415d630 con 0x7f5fd0004810 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd000ba60 con 0x7f5fe415cb70 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd0003a60 0x7f5fd0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe37fe700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fd0004810 0x7f5fd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe8d35700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 -- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 msgr2=0x7f5fd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 -- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fd0004810 msgr2=0x7f5fd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fd0004810 0x7f5fd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 -- 172.21.15.33:0/3179006930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0003570 con 0x7f5fd0003a60 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe37fe700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fd0004810 0x7f5fd0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe8d35700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fe415cb70 0x7f5fd00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd0003a60 0x7f5fd0003e70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009c30 tx=0x7f5fcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0899f435450f0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00e710 con 0x7f5fd0003a60 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc00e870 con 0x7f5fd0003a60 2023-10-06T02:46:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe2ffd700 1 -- 172.21.15.33:0/3179006930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00ec30 con 0x7f5fd0003a60 2023-10-06T02:46:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe9536700 1 -- 172.21.15.33:0/3179006930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd0003a60 msgr2=0x7f5fd0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe9536700 1 --2- 172.21.15.33:0/3179006930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fd0003a60 0x7f5fd0003e70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009c30 tx=0x7f5fcc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe8d35700 1 -- 172.21.15.33:0/3179006930 reap_dead start 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe9536700 1 -- 172.21.15.33:0/3179006930 shutdown_connections 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe9536700 1 -- 172.21.15.33:0/3179006930 >> 172.21.15.33:0/3179006930 conn(0x7f5fe4067a80 msgr2=0x7f5fe40cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe9536700 1 -- 172.21.15.33:0/3179006930 shutdown_connections 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:05.997+0000 7f5fe9536700 1 -- 172.21.15.33:0/3179006930 wait complete. 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.001+0000 7f5fe9536700 1 Processor -- start 2023-10-06T02:46:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.001+0000 7f5fe9536700 1 -- start start 2023-10-06T02:46:06.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.001+0000 7f5fe9536700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fd0003a60 0x7f5fe40b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 0x7f5fe4150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fe4150a90 0x7f5fe40c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4150fa0 con 0x7f5fe40b0a90 2023-10-06T02:46:06.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4151280 con 0x7f5fe4150a90 2023-10-06T02:46:06.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4151560 con 0x7f5fd0003a60 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 0x7f5fe4150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 0x7f5fe4150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54184/0 (socket says 172.21.15.33:54184) 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 -- 172.21.15.33:0/2248072258 learned_addr learned my addr 172.21.15.33:0/2248072258 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fd0003a60 0x7f5fe40b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe8d35700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fe4150a90 0x7f5fe40c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 -- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fd0003a60 msgr2=0x7f5fe40b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fd0003a60 0x7f5fe40b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 -- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fe4150a90 msgr2=0x7f5fe40c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fe4150a90 0x7f5fe40c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 -- 172.21.15.33:0/2248072258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4151840 con 0x7f5fe40b0a90 2023-10-06T02:46:06.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe37fe700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 0x7f5fe4150550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4010ca0 tx=0x7f5fd4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400e900 con 0x7f5fe40b0a90 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd400ea60 con 0x7f5fe40b0a90 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe4151b20 con 0x7f5fe40b0a90 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400ee00 con 0x7f5fe40b0a90 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe40c7210 con 0x7f5fe40b0a90 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe4071040 con 0x7f5fe40b0a90 2023-10-06T02:46:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.005+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5fd401c610 con 0x7f5fe40b0a90 2023-10-06T02:46:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.009+0000 7f5fdbfff700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5fc4060550 0x7f5fc4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.009+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f5fd407d1d0 con 0x7f5fe40b0a90 2023-10-06T02:46:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.009+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5fc4060550 0x7f5fc4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.009+0000 7f5fe3fff700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5fc4060550 0x7f5fc4062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0091e0 tx=0x7f5fcc01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.013+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5fd404d330 con 0x7f5fe40b0a90 2023-10-06T02:46:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.177+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fe40cda40 con 0x7f5fe40b0a90 2023-10-06T02:46:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.177+0000 7f5fdbfff700 1 -- 172.21.15.33:0/2248072258 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fd4050950 con 0x7f5fe40b0a90 2023-10-06T02:46:06.182 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.177+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5fc4060550 msgr2=0x7f5fc4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.177+0000 7f5fe9536700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5fc4060550 0x7f5fc4062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0091e0 tx=0x7f5fcc01a000 comp rx=0 tx=0).stop 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 msgr2=0x7f5fe4150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 0x7f5fe4150550 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4010ca0 tx=0x7f5fd4008040 comp rx=0 tx=0).stop 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 shutdown_connections 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fe4150a90 0x7f5fe40c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe40b0a90 0x7f5fe4150550 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5fc4060550 0x7f5fc4062a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 --2- 172.21.15.33:0/2248072258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fd0003a60 0x7f5fe40b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 >> 172.21.15.33:0/2248072258 conn(0x7f5fe4067a80 msgr2=0x7f5fe4153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 shutdown_connections 2023-10-06T02:46:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.181+0000 7f5fe9536700 1 -- 172.21.15.33:0/2248072258 wait complete. 2023-10-06T02:46:06.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:46:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.305+0000 7fd3ee853700 1 Processor -- start 2023-10-06T02:46:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.305+0000 7fd3ee853700 1 -- start start 2023-10-06T02:46:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd3e8158fc0 0x7fd3e815d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e815d830 con 0x7fd3e80b0530 2023-10-06T02:46:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e80b00a0 con 0x7fd3e8158bf0 2023-10-06T02:46:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e8067c10 con 0x7fd3e8158fc0 2023-10-06T02:46:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd3e80b0530 0x7fd3e815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39072/0 (socket says 172.21.15.33:39072) 2023-10-06T02:46:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e77fe700 1 -- 172.21.15.33:0/2327492139 learned_addr learned my addr 172.21.15.33:0/2327492139 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ecdf0700 1 --2- 172.21.15.33:0/2327492139 >> v2:172.21.15.167:3300/0 conn(0x7fd3e8158fc0 0x7fd3e815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e6ffd700 1 -- 172.21.15.33:0/2327492139 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850746707 0 0) 0x7fd3e815d830 con 0x7fd3e80b0530 2023-10-06T02:46:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e6ffd700 1 -- 172.21.15.33:0/2327492139 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d0003570 con 0x7fd3e80b0530 2023-10-06T02:46:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ecdf0700 1 -- 172.21.15.33:0/2327492139 >> v1:172.21.15.167:6789/0 conn(0x7fd3e8158bf0 legacy=0x7fd3e80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ecdf0700 1 -- 172.21.15.33:0/2327492139 >> v1:172.21.15.33:6789/0 conn(0x7fd3e80b0530 legacy=0x7fd3e815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ecdf0700 1 -- 172.21.15.33:0/2327492139 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e80cba20 con 0x7fd3e8158fc0 2023-10-06T02:46:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ecdf0700 1 --2- 172.21.15.33:0/2327492139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158fc0 0x7fd3e815d180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009e60 tx=0x7fd3e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88cc194dfc1fe90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e6ffd700 1 -- 172.21.15.33:0/2327492139 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000e070 con 0x7fd3e8158fc0 2023-10-06T02:46:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e6ffd700 1 -- 172.21.15.33:0/2327492139 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000bb60 con 0x7fd3e8158fc0 2023-10-06T02:46:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3e6ffd700 1 -- 172.21.15.33:0/2327492139 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0012430 con 0x7fd3e8158fc0 2023-10-06T02:46:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- 172.21.15.33:0/2327492139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158fc0 msgr2=0x7fd3e815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 --2- 172.21.15.33:0/2327492139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158fc0 0x7fd3e815d180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009e60 tx=0x7fd3e0009780 comp rx=0 tx=0).stop 2023-10-06T02:46:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- 172.21.15.33:0/2327492139 shutdown_connections 2023-10-06T02:46:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 --2- 172.21.15.33:0/2327492139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158fc0 0x7fd3e815d180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- 172.21.15.33:0/2327492139 >> 172.21.15.33:0/2327492139 conn(0x7fd3e80cfd70 msgr2=0x7fd3e80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- 172.21.15.33:0/2327492139 shutdown_connections 2023-10-06T02:46:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.309+0000 7fd3ee853700 1 -- 172.21.15.33:0/2327492139 wait complete. 2023-10-06T02:46:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 Processor -- start 2023-10-06T02:46:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 -- start start 2023-10-06T02:46:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158bf0 0x7fd3e814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e8158fc0 0x7fd3e815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e8159420 0x7fd3e80ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e81689d0 con 0x7fd3e8159420 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8168be0 con 0x7fd3e8158bf0 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8168ec0 con 0x7fd3e8158fc0 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ecdf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e8159420 0x7fd3e80ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158bf0 0x7fd3e814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ecdf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e8159420 0x7fd3e80ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54186/0 (socket says 172.21.15.33:54186) 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ecdf0700 1 -- 172.21.15.33:0/705848020 learned_addr learned my addr 172.21.15.33:0/705848020 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e77fe700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e8158fc0 0x7fd3e815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 -- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e8158fc0 msgr2=0x7fd3e815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e8158fc0 0x7fd3e815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 -- 172.21.15.33:0/705848020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e8159420 msgr2=0x7fd3e80ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e8159420 0x7fd3e80ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 -- 172.21.15.33:0/705848020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0009a20 con 0x7fd3e8158bf0 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3e7fff700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158bf0 0x7fd3e814f020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c490 tx=0x7fd3dc007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc00ce40 con 0x7fd3e8158bf0 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc011590 con 0x7fd3e8158bf0 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e81691a0 con 0x7fd3e8158bf0 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.313+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc011950 con 0x7fd3e8158bf0 2023-10-06T02:46:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.317+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e8169760 con 0x7fd3e8158bf0 2023-10-06T02:46:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.317+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e8169980 con 0x7fd3e8158bf0 2023-10-06T02:46:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.317+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd3dc011cc0 con 0x7fd3e8158bf0 2023-10-06T02:46:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.317+0000 7fd3cffff700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3d4060550 0x7fd3d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.317+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fd3dc07bfb0 con 0x7fd3e8158bf0 2023-10-06T02:46:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.317+0000 7fd3e77fe700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3d4060550 0x7fd3d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.321+0000 7fd3e77fe700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3d4060550 0x7fd3d4062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80aeb40 tx=0x7fd3d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.325+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd3dc04c110 con 0x7fd3e8158bf0 2023-10-06T02:46:06.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.493+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3e80710b0 con 0x7fd3e8158bf0 2023-10-06T02:46:06.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.493+0000 7fd3cffff700 1 -- 172.21.15.33:0/705848020 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3dc017020 con 0x7fd3e8158bf0 2023-10-06T02:46:06.498 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:46:06.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3d4060550 msgr2=0x7fd3d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3d4060550 0x7fd3d4062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80aeb40 tx=0x7fd3d8009040 comp rx=0 tx=0).stop 2023-10-06T02:46:06.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158bf0 msgr2=0x7fd3e814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158bf0 0x7fd3e814f020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c490 tx=0x7fd3dc007ea0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 shutdown_connections 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3d4060550 0x7fd3d4062a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e8158fc0 0x7fd3e815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e8159420 0x7fd3e80ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 --2- 172.21.15.33:0/705848020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e8158bf0 0x7fd3e814f020 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 >> 172.21.15.33:0/705848020 conn(0x7fd3e80cfd70 msgr2=0x7fd3e80cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 shutdown_connections 2023-10-06T02:46:06.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.497+0000 7fd3ee853700 1 -- 172.21.15.33:0/705848020 wait complete. 2023-10-06T02:46:06.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:46:06.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351e15700 1 Processor -- start 2023-10-06T02:46:06.633 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351e15700 1 -- start start 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351e15700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f334c0cf250 0x7f334c163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351e15700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0cd7f0 con 0x7f334c159030 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351e15700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0cddb0 con 0x7f334c158c60 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351e15700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c159480 con 0x7f334c0cf250 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f334c159030 0x7f334c0caae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39078/0 (socket says 172.21.15.33:39078) 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334bfff700 1 -- 172.21.15.33:0/3169064273 learned_addr learned my addr 172.21.15.33:0/3169064273 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351614700 1 --2- 172.21.15.33:0/3169064273 >> v2:172.21.15.167:3300/0 conn(0x7f334c0cf250 0x7f334c163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628827457 0 0) 0x7f334c0cd7f0 con 0x7f334c159030 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3334003570 con 0x7f334c159030 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216509620 0 0) 0x7f334c0cddb0 con 0x7f334c158c60 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f334c0cd7f0 con 0x7f334c158c60 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351614700 1 -- 172.21.15.33:0/3169064273 >> v1:172.21.15.167:6789/0 conn(0x7f334c158c60 legacy=0x7f334c15cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351614700 1 -- 172.21.15.33:0/3169064273 >> v1:172.21.15.33:6789/0 conn(0x7f334c159030 legacy=0x7f334c0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351614700 1 -- 172.21.15.33:0/3169064273 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c15d630 con 0x7f334c0cf250 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f3351614700 1 --2- 172.21.15.33:0/3169064273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c0cf250 0x7f334c163eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3344009e60 tx=0x7f3344009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d848bb80bcfca4a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.629+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400e070 con 0x7f334c0cf250 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334400bab0 con 0x7f334c0cf250 2023-10-06T02:46:06.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f334b7fe700 1 -- 172.21.15.33:0/3169064273 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400be30 con 0x7f334c0cf250 2023-10-06T02:46:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 -- 172.21.15.33:0/3169064273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c0cf250 msgr2=0x7f334c163eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 --2- 172.21.15.33:0/3169064273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c0cf250 0x7f334c163eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3344009e60 tx=0x7f3344009780 comp rx=0 tx=0).stop 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 -- 172.21.15.33:0/3169064273 shutdown_connections 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 --2- 172.21.15.33:0/3169064273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c0cf250 0x7f334c163eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 -- 172.21.15.33:0/3169064273 >> 172.21.15.33:0/3169064273 conn(0x7f334c067a80 msgr2=0x7f334c0cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 -- 172.21.15.33:0/3169064273 shutdown_connections 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 -- 172.21.15.33:0/3169064273 wait complete. 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 Processor -- start 2023-10-06T02:46:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.633+0000 7f3351e15700 1 -- start start 2023-10-06T02:46:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 0x7f334c14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c158c60 0x7f334c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f334c159030 0x7f334c159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0b0980 con 0x7f334c0cf250 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c15ebf0 con 0x7f334c158c60 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c15eed0 con 0x7f334c159030 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 0x7f334c14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 0x7f334c14e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54194/0 (socket says 172.21.15.33:54194) 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 -- 172.21.15.33:0/3844248655 learned_addr learned my addr 172.21.15.33:0/3844248655 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f334bfff700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c158c60 0x7f334c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351614700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f334c159030 0x7f334c159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 -- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f334c159030 msgr2=0x7f334c159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f334c159030 0x7f334c159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 -- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c158c60 msgr2=0x7f334c14e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c158c60 0x7f334c14e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 -- 172.21.15.33:0/3844248655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3344009a20 con 0x7f334c0cf250 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3350e13700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 0x7f334c14e760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c790 tx=0x7f333c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c010510 con 0x7f334c0cf250 2023-10-06T02:46:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c010670 con 0x7f334c0cf250 2023-10-06T02:46:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c15f1b0 con 0x7f334c0cf250 2023-10-06T02:46:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c010a10 con 0x7f334c0cf250 2023-10-06T02:46:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c14ee80 con 0x7f334c0cf250 2023-10-06T02:46:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.637+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c071040 con 0x7f334c0cf250 2023-10-06T02:46:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.641+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f333c01b3b0 con 0x7f334c0cf250 2023-10-06T02:46:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.641+0000 7f3348ff9700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3338060550 0x7f3338062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.641+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f333c07c4d0 con 0x7f334c0cf250 2023-10-06T02:46:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.641+0000 7f334bfff700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3338060550 0x7f3338062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.641+0000 7f334bfff700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3338060550 0x7f3338062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3340007fd0 tx=0x7f3340009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.649+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f333c04c630 con 0x7f334c0cf250 2023-10-06T02:46:06.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.809+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f334c15d910 con 0x7f334c0cf250 2023-10-06T02:46:06.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.809+0000 7f3348ff9700 1 -- 172.21.15.33:0/3844248655 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v32) v1 ==== 103+0+11 (secure 0 0 0) 0x7f333c0170a0 con 0x7f334c0cf250 2023-10-06T02:46:06.814 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3338060550 msgr2=0x7f3338062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3338060550 0x7f3338062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3340007fd0 tx=0x7f3340009040 comp rx=0 tx=0).stop 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 msgr2=0x7f334c14e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 0x7f334c14e760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c790 tx=0x7f333c007e10 comp rx=0 tx=0).stop 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 shutdown_connections 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3338060550 0x7f3338062a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f334c159030 0x7f334c159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c158c60 0x7f334c14e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 --2- 172.21.15.33:0/3844248655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f334c0cf250 0x7f334c14e760 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 >> 172.21.15.33:0/3844248655 conn(0x7f334c067a80 msgr2=0x7f334c0ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 shutdown_connections 2023-10-06T02:46:06.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.813+0000 7f3351e15700 1 -- 172.21.15.33:0/3844248655 wait complete. 2023-10-06T02:46:06.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.929+0000 7fd3eaec6700 1 Processor -- start 2023-10-06T02:46:06.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.929+0000 7fd3eaec6700 1 -- start start 2023-10-06T02:46:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd3e40aec50 0x7fd3e40cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40684d0 0x7fd3e4068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e4068e60 con 0x7fd3e40b2b10 2023-10-06T02:46:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e40b2070 con 0x7fd3e40aec50 2023-10-06T02:46:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e41592e0 con 0x7fd3e40684d0 2023-10-06T02:46:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3ea6c5700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd3e40b2b10 0x7fd3e40b2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47166/0 (socket says 172.21.15.33:47166) 2023-10-06T02:46:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3ea6c5700 1 -- 172.21.15.33:0/1987871365 learned_addr learned my addr 172.21.15.33:0/1987871365 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e96c3700 1 --2- 172.21.15.33:0/1987871365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40684d0 0x7fd3e4068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e9ec4700 1 --2- 172.21.15.33:0/1987871365 >> v2:172.21.15.167:3300/0 conn(0x7fd3e40aec50 0x7fd3e40cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e8ec2700 1 -- 172.21.15.33:0/1987871365 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349658971 0 0) 0x7fd3e4068e60 con 0x7fd3e40b2b10 2023-10-06T02:46:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e8ec2700 1 -- 172.21.15.33:0/1987871365 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d0003570 con 0x7fd3e40b2b10 2023-10-06T02:46:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e96c3700 1 -- 172.21.15.33:0/1987871365 >> v2:172.21.15.167:3300/0 conn(0x7fd3e40aec50 msgr2=0x7fd3e40cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e96c3700 1 --2- 172.21.15.33:0/1987871365 >> v2:172.21.15.167:3300/0 conn(0x7fd3e40aec50 0x7fd3e40cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e96c3700 1 -- 172.21.15.33:0/1987871365 >> v1:172.21.15.167:6789/0 conn(0x7fd3e40b2b10 legacy=0x7fd3e40b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e96c3700 1 -- 172.21.15.33:0/1987871365 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e40b5ae0 con 0x7fd3e40684d0 2023-10-06T02:46:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e9ec4700 1 --2- 172.21.15.33:0/1987871365 >> v2:172.21.15.167:3300/0 conn(0x7fd3e40aec50 0x7fd3e40cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e96c3700 1 --2- 172.21.15.33:0/1987871365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40684d0 0x7fd3e4068920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00098e0 tx=0x7fd3e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511a677e7498ffdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e8ec2700 1 -- 172.21.15.33:0/1987871365 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0011630 con 0x7fd3e40684d0 2023-10-06T02:46:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e8ec2700 1 -- 172.21.15.33:0/1987871365 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e0011790 con 0x7fd3e40684d0 2023-10-06T02:46:06.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3e8ec2700 1 -- 172.21.15.33:0/1987871365 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0011b10 con 0x7fd3e40684d0 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- 172.21.15.33:0/1987871365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40684d0 msgr2=0x7fd3e4068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/1987871365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40684d0 0x7fd3e4068920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00098e0 tx=0x7fd3e00091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- 172.21.15.33:0/1987871365 shutdown_connections 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/1987871365 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40684d0 0x7fd3e4068920 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/1987871365 >> v2:172.21.15.167:3300/0 conn(0x7fd3e40aec50 0x7fd3e40cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- 172.21.15.33:0/1987871365 >> 172.21.15.33:0/1987871365 conn(0x7fd3e4067a40 msgr2=0x7fd3e40b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:06.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- 172.21.15.33:0/1987871365 shutdown_connections 2023-10-06T02:46:06.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.933+0000 7fd3eaec6700 1 -- 172.21.15.33:0/1987871365 wait complete. 2023-10-06T02:46:06.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 Processor -- start 2023-10-06T02:46:06.941 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 -- start start 2023-10-06T02:46:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e40684d0 0x7fd3e40c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 0x7fd3e41477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40b2b10 0x7fd3e415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40cb570 con 0x7fd3e40aec50 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40ca6e0 con 0x7fd3e40684d0 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e40ca960 con 0x7fd3e40b2b10 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 0x7fd3e41477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 0x7fd3e41477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54202/0 (socket says 172.21.15.33:54202) 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 -- 172.21.15.33:0/4078126564 learned_addr learned my addr 172.21.15.33:0/4078126564 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e9ec4700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e40684d0 0x7fd3e40c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.943 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3ea6c5700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40b2b10 0x7fd3e415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 -- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40b2b10 msgr2=0x7fd3e415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40b2b10 0x7fd3e415d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 -- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e40684d0 msgr2=0x7fd3e40c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:06.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e40684d0 0x7fd3e40c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 -- 172.21.15.33:0/4078126564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009420 con 0x7fd3e40aec50 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3e96c3700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 0x7fd3e41477b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009ed0 tx=0x7fd3e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0011900 con 0x7fd3e40aec50 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e0009420 con 0x7fd3e40aec50 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e40caf20 con 0x7fd3e40aec50 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e001a430 con 0x7fd3e40aec50 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.937+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e001a5e0 con 0x7fd3e40aec50 2023-10-06T02:46:06.945 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.941+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e40cb720 con 0x7fd3e40aec50 2023-10-06T02:46:06.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.941+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd3e001aa10 con 0x7fd3e40aec50 2023-10-06T02:46:06.946 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.941+0000 7fd3d9ffb700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3c8060550 0x7fd3c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:06.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.941+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fd3e007e700 con 0x7fd3e40aec50 2023-10-06T02:46:06.947 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.941+0000 7fd3e9ec4700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3c8060550 0x7fd3c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:06.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.945+0000 7fd3e9ec4700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3c8060550 0x7fd3c8062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009f40 tx=0x7fd3d40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:06.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:06.949+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd3e40cb720 con 0x7fd3e40aec50 2023-10-06T02:46:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.113+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3e40cb720 con 0x7fd3e40aec50 2023-10-06T02:46:07.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3d9ffb700 1 -- 172.21.15.33:0/4078126564 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fd3e40cb720 con 0x7fd3e40aec50 2023-10-06T02:46:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:07.121 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:05.355164+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3c8060550 msgr2=0x7fd3c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3c8060550 0x7fd3c8062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009f40 tx=0x7fd3d40091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 msgr2=0x7fd3e41477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 0x7fd3e41477b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009ed0 tx=0x7fd3e0017000 comp rx=0 tx=0).stop 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 shutdown_connections 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd3c8060550 0x7fd3c8062a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3e40b2b10 0x7fd3e415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40aec50 0x7fd3e41477b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 --2- 172.21.15.33:0/4078126564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3e40684d0 0x7fd3e40c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 >> 172.21.15.33:0/4078126564 conn(0x7fd3e4067a40 msgr2=0x7fd3e40afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 shutdown_connections 2023-10-06T02:46:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.117+0000 7fd3eaec6700 1 -- 172.21.15.33:0/4078126564 wait complete. 2023-10-06T02:46:07.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:46:07.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:07.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 Processor -- start 2023-10-06T02:46:07.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 -- start start 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54a4157280 0x7f54a40aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f54a40679c0 0x7f54a4067e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157650 0x7f54a4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a40c6c10 con 0x7f54a4157280 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a4068380 con 0x7f54a40679c0 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a949e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4157ff0 con 0x7f54a4157650 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.333+0000 7f54a2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54a4157280 0x7f54a40aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f54a4157280 0x7f54a40aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54206/0 (socket says 172.21.15.33:54206) 2023-10-06T02:46:07.340 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 -- 172.21.15.33:0/582745582 learned_addr learned my addr 172.21.15.33:0/582745582 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:07.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a37fe700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157650 0x7f54a4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.341 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a27fc700 1 --2- 172.21.15.33:0/582745582 >> v2:172.21.15.167:3300/0 conn(0x7f54a40679c0 0x7f54a4067e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 -- 172.21.15.33:0/582745582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157650 msgr2=0x7f54a4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157650 0x7f54a4157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 -- 172.21.15.33:0/582745582 >> v2:172.21.15.167:3300/0 conn(0x7f54a40679c0 msgr2=0x7f54a4067e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 --2- 172.21.15.33:0/582745582 >> v2:172.21.15.167:3300/0 conn(0x7f54a40679c0 0x7f54a4067e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 -- 172.21.15.33:0/582745582 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40685d0 con 0x7f54a4157280 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a37fe700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157650 0x7f54a4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a2ffd700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157280 0x7f54a40aadd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54a40c7210 tx=0x7f5498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7918e6e8bfc59b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a1ffb700 1 -- 172.21.15.33:0/582745582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800b640 con 0x7f54a4157280 2023-10-06T02:46:07.342 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a1ffb700 1 -- 172.21.15.33:0/582745582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549800b7a0 con 0x7f54a4157280 2023-10-06T02:46:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a1ffb700 1 -- 172.21.15.33:0/582745582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800bb60 con 0x7f54a4157280 2023-10-06T02:46:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 -- 172.21.15.33:0/582745582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157280 msgr2=0x7f54a40aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157280 0x7f54a40aadd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54a40c7210 tx=0x7f5498009250 comp rx=0 tx=0).stop 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 -- 172.21.15.33:0/582745582 shutdown_connections 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157650 0x7f54a4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 --2- 172.21.15.33:0/582745582 >> v2:172.21.15.167:3300/0 conn(0x7f54a40679c0 0x7f54a4067e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 --2- 172.21.15.33:0/582745582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157280 0x7f54a40aadd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 -- 172.21.15.33:0/582745582 >> 172.21.15.33:0/582745582 conn(0x7f54a40c83f0 msgr2=0x7f54a40c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 -- 172.21.15.33:0/582745582 shutdown_connections 2023-10-06T02:46:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 -- 172.21.15.33:0/582745582 wait complete. 2023-10-06T02:46:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.337+0000 7f54a949e700 1 Processor -- start 2023-10-06T02:46:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 -- start start 2023-10-06T02:46:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54a40679c0 0x7f54a41556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157280 0x7f54a4169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 0x7f54a4168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.348 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4155c00 con 0x7f54a4157650 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40c41a0 con 0x7f54a40679c0 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a949e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40c33c0 con 0x7f54a4157280 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 0x7f54a4168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54a40679c0 0x7f54a41556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 0x7f54a4168dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54218/0 (socket says 172.21.15.33:54218) 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a37fe700 1 -- 172.21.15.33:0/2191838364 learned_addr learned my addr 172.21.15.33:0/2191838364 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.341+0000 7f54a27fc700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157280 0x7f54a4169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a37fe700 1 -- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157280 msgr2=0x7f54a4169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a37fe700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157280 0x7f54a4169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a37fe700 1 -- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54a40679c0 msgr2=0x7f54a41556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a37fe700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54a40679c0 0x7f54a41556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a37fe700 1 -- 172.21.15.33:0/2191838364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c009a20 con 0x7f54a4157650 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a37fe700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 0x7f54a4168dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f50 tx=0x7f549c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.350 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00cb50 con 0x7f54a4157650 2023-10-06T02:46:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00f040 con 0x7f54a4157650 2023-10-06T02:46:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c014480 con 0x7f54a4157650 2023-10-06T02:46:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a949e700 1 -- 172.21.15.33:0/2191838364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54980094f0 con 0x7f54a4157650 2023-10-06T02:46:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f54a949e700 1 -- 172.21.15.33:0/2191838364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a4169c80 con 0x7f54a4157650 2023-10-06T02:46:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.345+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c0052b0 con 0x7f54a4157650 2023-10-06T02:46:07.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.349+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f549c003020 con 0x7f54a4157650 2023-10-06T02:46:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.349+0000 7f548affd700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5490060550 0x7f5490062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.349+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f549c07d480 con 0x7f54a4157650 2023-10-06T02:46:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.349+0000 7f54a2ffd700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5490060550 0x7f5490062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.349+0000 7f54a2ffd700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5490060550 0x7f5490062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54a40c7210 tx=0x7f5498017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.353+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f549c04d5e0 con 0x7f54a4157650 2023-10-06T02:46:07.517 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.513+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f547c005ae0 con 0x7f54a4157650 2023-10-06T02:46:07.518 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.513+0000 7f548affd700 1 -- 172.21.15.33:0/2191838364 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f549c050c00 con 0x7f54a4157650 2023-10-06T02:46:07.518 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:07.520 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5490060550 msgr2=0x7f5490062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5490060550 0x7f5490062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54a40c7210 tx=0x7f5498017000 comp rx=0 tx=0).stop 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 msgr2=0x7f54a4168dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 0x7f54a4168dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f50 tx=0x7f549c00b040 comp rx=0 tx=0).stop 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 shutdown_connections 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5490060550 0x7f5490062a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54a4157280 0x7f54a4169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54a4157650 0x7f54a4168dc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 --2- 172.21.15.33:0/2191838364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54a40679c0 0x7f54a41556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 >> 172.21.15.33:0/2191838364 conn(0x7f54a40c83f0 msgr2=0x7f54a4156470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 shutdown_connections 2023-10-06T02:46:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.517+0000 7f5488ff9700 1 -- 172.21.15.33:0/2191838364 wait complete. 2023-10-06T02:46:07.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 Processor -- start 2023-10-06T02:46:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- start start 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe9441592e0 0x7fe944159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe9440c7130 0x7fe944068460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9440cad70 con 0x7fe944159a30 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9440ca160 con 0x7fe9441592e0 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9440ca440 con 0x7fe9440c7130 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe9441592e0 0x7fe944159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe9441592e0 0x7fe944159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54232/0 (socket says 172.21.15.33:54232) 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 -- 172.21.15.33:0/3327582204 learned_addr learned my addr 172.21.15.33:0/3327582204 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94924f700 1 --2- 172.21.15.33:0/3327582204 >> v2:172.21.15.167:3300/0 conn(0x7fe9440c7130 0x7fe944068460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 -- 172.21.15.33:0/3327582204 >> v2:172.21.15.167:3300/0 conn(0x7fe9440c7130 msgr2=0x7fe944068460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 --2- 172.21.15.33:0/3327582204 >> v2:172.21.15.167:3300/0 conn(0x7fe9440c7130 0x7fe944068460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 -- 172.21.15.33:0/3327582204 >> v1:172.21.15.167:6789/0 conn(0x7fe944159a30 legacy=0x7fe944158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 -- 172.21.15.33:0/3327582204 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440ca720 con 0x7fe9441592e0 2023-10-06T02:46:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94924f700 1 --2- 172.21.15.33:0/3327582204 >> v2:172.21.15.167:3300/0 conn(0x7fe9440c7130 0x7fe944068460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe943fff700 1 --2- 172.21.15.33:0/3327582204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9441592e0 0x7fe944159e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe9440c9110 tx=0x7fe930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281ed249de851232 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe9437fe700 1 -- 172.21.15.33:0/3327582204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000b860 con 0x7fe9441592e0 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe9437fe700 1 -- 172.21.15.33:0/3327582204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93000b9c0 con 0x7fe9441592e0 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe9437fe700 1 -- 172.21.15.33:0/3327582204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000bd60 con 0x7fe9441592e0 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- 172.21.15.33:0/3327582204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9441592e0 msgr2=0x7fe944159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 --2- 172.21.15.33:0/3327582204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9441592e0 0x7fe944159e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe9440c9110 tx=0x7fe930009250 comp rx=0 tx=0).stop 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- 172.21.15.33:0/3327582204 shutdown_connections 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 --2- 172.21.15.33:0/3327582204 >> v2:172.21.15.167:3300/0 conn(0x7fe9440c7130 0x7fe944068460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 --2- 172.21.15.33:0/3327582204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9441592e0 0x7fe944159e00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- 172.21.15.33:0/3327582204 >> 172.21.15.33:0/3327582204 conn(0x7fe9440cc520 msgr2=0x7fe9440c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- 172.21.15.33:0/3327582204 shutdown_connections 2023-10-06T02:46:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.641+0000 7fe94acb2700 1 -- 172.21.15.33:0/3327582204 wait complete. 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 Processor -- start 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 -- start start 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9440c7130 0x7fe944169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe944159a30 0x7fe94414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 0x7fe94406aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe944163950 con 0x7fe9440aec60 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94416c820 con 0x7fe944159a30 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94acb2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440afda0 con 0x7fe9440c7130 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 0x7fe94406aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe943fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe944159a30 0x7fe94414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 0x7fe94406aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54244/0 (socket says 172.21.15.33:54244) 2023-10-06T02:46:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe948a4e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9440c7130 0x7fe944169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 -- 172.21.15.33:0/2827409008 learned_addr learned my addr 172.21.15.33:0/2827409008 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:07.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 -- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9440c7130 msgr2=0x7fe944169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9440c7130 0x7fe944169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 -- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe944159a30 msgr2=0x7fe94414e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe944159a30 0x7fe94414e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 -- 172.21.15.33:0/2827409008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9300094f0 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.645+0000 7fe94924f700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 0x7fe94406aa10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00d180 tx=0x7fe93c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00dcc0 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c00f040 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94416a370 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c015440 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9440af470 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9440704c0 con 0x7fe9440aec60 2023-10-06T02:46:07.657 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe93c01e3b0 con 0x7fe9440aec60 2023-10-06T02:46:07.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe940ff9700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe928060580 0x7fe928062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fe93c07d130 con 0x7fe9440aec60 2023-10-06T02:46:07.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.649+0000 7fe948a4e700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe928060580 0x7fe928062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.658 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.653+0000 7fe948a4e700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe928060580 0x7fe928062a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9440aea00 tx=0x7fe938009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.662 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.657+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe93c04d1f0 con 0x7fe9440aec60 2023-10-06T02:46:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9440abe80 con 0x7fe9440aec60 2023-10-06T02:46:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe940ff9700 1 -- 172.21.15.33:0/2827409008 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe93c050810 con 0x7fe9440aec60 2023-10-06T02:46:07.825 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe928060580 msgr2=0x7fe928062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe928060580 0x7fe928062a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9440aea00 tx=0x7fe938009040 comp rx=0 tx=0).stop 2023-10-06T02:46:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 msgr2=0x7fe94406aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 0x7fe94406aa10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00d180 tx=0x7fe93c009a50 comp rx=0 tx=0).stop 2023-10-06T02:46:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 shutdown_connections 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9440aec60 0x7fe94406aa10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe944159a30 0x7fe94414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.821+0000 7fe94acb2700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe928060580 0x7fe928062a30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.825+0000 7fe94acb2700 1 --2- 172.21.15.33:0/2827409008 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9440c7130 0x7fe944169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.825+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 >> 172.21.15.33:0/2827409008 conn(0x7fe9440cc520 msgr2=0x7fe9441580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.825+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 shutdown_connections 2023-10-06T02:46:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.825+0000 7fe94acb2700 1 -- 172.21.15.33:0/2827409008 wait complete. 2023-10-06T02:46:07.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:46:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.945+0000 7f96bce1f700 1 Processor -- start 2023-10-06T02:46:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.945+0000 7f96bce1f700 1 -- start start 2023-10-06T02:46:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 0x7f96b8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b80688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80ce580 con 0x7f96b8159030 2023-10-06T02:46:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b80cf620 con 0x7f96b8158c60 2023-10-06T02:46:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80ce860 con 0x7f96b80cf7c0 2023-10-06T02:46:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b80688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b77fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 0x7f96b8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b80688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:40162/0 (socket says 172.21.15.33:40162) 2023-10-06T02:46:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b77fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 0x7f96b8068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:47750/0 (socket says 172.21.15.33:47750) 2023-10-06T02:46:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 -- 172.21.15.33:0/1993060717 learned_addr learned my addr 172.21.15.33:0/1993060717 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b7fff700 1 --1- 172.21.15.33:0/1993060717 >> v1:172.21.15.167:6789/0 conn(0x7f96b8159030 0x7f96b815ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47190/0 (socket says 172.21.15.33:47190) 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 -- 172.21.15.33:0/1993060717 >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 msgr2=0x7f96b8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 --2- 172.21.15.33:0/1993060717 >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 0x7f96b8068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 -- 172.21.15.33:0/1993060717 >> v1:172.21.15.167:6789/0 conn(0x7f96b8159030 legacy=0x7f96b815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b67fc700 1 -- 172.21.15.33:0/1993060717 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640501343 0 0) 0x7f96b80ce580 con 0x7f96b8159030 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 -- 172.21.15.33:0/1993060717 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80ceb40 con 0x7f96b80cf7c0 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b77fe700 1 --2- 172.21.15.33:0/1993060717 >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 0x7f96b8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b6ffd700 1 --2- 172.21.15.33:0/1993060717 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b80688d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0099b0 tx=0x7f96ac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8118109321725374 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b67fc700 1 -- 172.21.15.33:0/1993060717 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac009ea0 con 0x7f96b80cf7c0 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b67fc700 1 -- 172.21.15.33:0/1993060717 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac00b860 con 0x7f96b80cf7c0 2023-10-06T02:46:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96b67fc700 1 -- 172.21.15.33:0/1993060717 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00bc00 con 0x7f96b80cf7c0 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- 172.21.15.33:0/1993060717 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 msgr2=0x7f96b80688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 --2- 172.21.15.33:0/1993060717 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b80688d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0099b0 tx=0x7f96ac009250 comp rx=0 tx=0).stop 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- 172.21.15.33:0/1993060717 shutdown_connections 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 --2- 172.21.15.33:0/1993060717 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b80688d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 --2- 172.21.15.33:0/1993060717 >> v2:172.21.15.167:3300/0 conn(0x7f96b8158c60 0x7f96b8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- 172.21.15.33:0/1993060717 >> 172.21.15.33:0/1993060717 conn(0x7f96b8067a40 msgr2=0x7f96b80b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- 172.21.15.33:0/1993060717 shutdown_connections 2023-10-06T02:46:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.949+0000 7f96bce1f700 1 -- 172.21.15.33:0/1993060717 wait complete. 2023-10-06T02:46:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 Processor -- start 2023-10-06T02:46:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 -- start start 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 0x7f96b80b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b8159030 0x7f96b80b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b8152970 con 0x7f96b8158c60 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b814c090 con 0x7f96b8159030 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96bce1f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b816d140 con 0x7f96b80cf7c0 2023-10-06T02:46:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 0x7f96b80b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 0x7f96b80b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54250/0 (socket says 172.21.15.33:54250) 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 -- 172.21.15.33:0/4152828970 learned_addr learned my addr 172.21.15.33:0/4152828970 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b77fe700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b7fff700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b8159030 0x7f96b80b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 -- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 msgr2=0x7f96b814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b814b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 -- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b8159030 msgr2=0x7f96b80b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b8159030 0x7f96b80b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 -- 172.21.15.33:0/4152828970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80093c0 con 0x7f96b8158c60 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b7fff700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b8159030 0x7f96b80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.953+0000 7f96b6ffd700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 0x7f96b80b4370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96ac006fd0 tx=0x7f96ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00bbe0 con 0x7f96b8158c60 2023-10-06T02:46:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac011420 con 0x7f96b8158c60 2023-10-06T02:46:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0117e0 con 0x7f96b8158c60 2023-10-06T02:46:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac0094f0 con 0x7f96b8158c60 2023-10-06T02:46:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b816caf0 con 0x7f96b8158c60 2023-10-06T02:46:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b8071040 con 0x7f96b8158c60 2023-10-06T02:46:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.957+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f96ac011bd0 con 0x7f96b8158c60 2023-10-06T02:46:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.961+0000 7f969f7fe700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96a4060580 0x7f96a4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.961+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f96ac07e560 con 0x7f96b8158c60 2023-10-06T02:46:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.961+0000 7f96b77fe700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96a4060580 0x7f96a4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.961+0000 7f96b77fe700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96a4060580 0x7f96a4062a30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009f30 tx=0x7f96a8009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:07.969+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f96ac04e620 con 0x7f96b8158c60 2023-10-06T02:46:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96b816d280 con 0x7f96b8158c60 2023-10-06T02:46:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f969f7fe700 1 -- 172.21.15.33:0/4152828970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96ac051c40 con 0x7f96b8158c60 2023-10-06T02:46:08.137 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96a4060580 msgr2=0x7f96a4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96a4060580 0x7f96a4062a30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009f30 tx=0x7f96a8009120 comp rx=0 tx=0).stop 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 msgr2=0x7f96b80b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 0x7f96b80b4370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96ac006fd0 tx=0x7f96ac017000 comp rx=0 tx=0).stop 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 shutdown_connections 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b8159030 0x7f96b80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96b8158c60 0x7f96b80b4370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96a4060580 0x7f96a4062a30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 --2- 172.21.15.33:0/4152828970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b80cf7c0 0x7f96b814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.133+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 >> 172.21.15.33:0/4152828970 conn(0x7f96b8067a40 msgr2=0x7f96b80afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.137+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 shutdown_connections 2023-10-06T02:46:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.137+0000 7f96bce1f700 1 -- 172.21.15.33:0/4152828970 wait complete. 2023-10-06T02:46:08.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:46:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.261+0000 7f6b7a36c700 1 Processor -- start 2023-10-06T02:46:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.261+0000 7f6b7a36c700 1 -- start start 2023-10-06T02:46:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6b740cfe10 0x7f6b740d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b740cf0c0 0x7f6b74158670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cf540 0x7f6b74158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b740cec30 con 0x7f6b740cf0c0 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b741590f0 con 0x7f6b740cfe10 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7415d710 con 0x7f6b740cf540 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b740cf0c0 0x7f6b74158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6b740cf0c0 0x7f6b74158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54264/0 (socket says 172.21.15.33:54264) 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 -- 172.21.15.33:0/179307449 learned_addr learned my addr 172.21.15.33:0/179307449 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b79b6b700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cf540 0x7f6b74158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7936a700 1 --2- 172.21.15.33:0/179307449 >> v2:172.21.15.167:3300/0 conn(0x7f6b740cfe10 0x7f6b740d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 -- 172.21.15.33:0/179307449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cf540 msgr2=0x7f6b74158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cf540 0x7f6b74158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 -- 172.21.15.33:0/179307449 >> v2:172.21.15.167:3300/0 conn(0x7f6b740cfe10 msgr2=0x7f6b740d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 --2- 172.21.15.33:0/179307449 >> v2:172.21.15.167:3300/0 conn(0x7f6b740cfe10 0x7f6b740d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 -- 172.21.15.33:0/179307449 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740ce300 con 0x7f6b740cf0c0 2023-10-06T02:46:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b79b6b700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cf540 0x7f6b74158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b78b69700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b74158670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b7406f2d0 tx=0x7f6b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6f46a0275211fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b67fff700 1 -- 172.21.15.33:0/179307449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800b860 con 0x7f6b740cf0c0 2023-10-06T02:46:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b67fff700 1 -- 172.21.15.33:0/179307449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6800b9c0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b67fff700 1 -- 172.21.15.33:0/179307449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800bd60 con 0x7f6b740cf0c0 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- 172.21.15.33:0/179307449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 msgr2=0x7f6b74158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b74158670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b7406f2d0 tx=0x7f6b68009250 comp rx=0 tx=0).stop 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- 172.21.15.33:0/179307449 shutdown_connections 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cf540 0x7f6b74158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/179307449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b74158670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/179307449 >> v2:172.21.15.167:3300/0 conn(0x7f6b740cfe10 0x7f6b740d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- 172.21.15.33:0/179307449 >> 172.21.15.33:0/179307449 conn(0x7f6b74066fb0 msgr2=0x7f6b74067380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- 172.21.15.33:0/179307449 shutdown_connections 2023-10-06T02:46:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.265+0000 7f6b7a36c700 1 -- 172.21.15.33:0/179307449 wait complete. 2023-10-06T02:46:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 Processor -- start 2023-10-06T02:46:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 -- start start 2023-10-06T02:46:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b740cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740cf540 0x7f6b740c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cfe10 0x7f6b74154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b740c5b20 con 0x7f6b740cf0c0 2023-10-06T02:46:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b740c4f10 con 0x7f6b740cf540 2023-10-06T02:46:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7a36c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b740c51f0 con 0x7f6b740cfe10 2023-10-06T02:46:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b740cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b740cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54268/0 (socket says 172.21.15.33:54268) 2023-10-06T02:46:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 -- 172.21.15.33:0/563507974 learned_addr learned my addr 172.21.15.33:0/563507974 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b78b69700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740cf540 0x7f6b740c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b79b6b700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cfe10 0x7f6b74154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 -- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cfe10 msgr2=0x7f6b74154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cfe10 0x7f6b74154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 -- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740cf540 msgr2=0x7f6b740c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740cf540 0x7f6b740c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.269+0000 7f6b7936a700 1 -- 172.21.15.33:0/563507974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680094f0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b7936a700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b740cc2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000d860 tx=0x7f6b70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700125f0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70012750 con 0x7f6b740cf0c0 2023-10-06T02:46:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b740c54d0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70012b10 con 0x7f6b740cf0c0 2023-10-06T02:46:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b740cc830 con 0x7f6b740cf0c0 2023-10-06T02:46:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b74071040 con 0x7f6b740cf0c0 2023-10-06T02:46:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.273+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6b7001e460 con 0x7f6b740cf0c0 2023-10-06T02:46:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.277+0000 7f6b657fa700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6b58060550 0x7f6b58062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.277+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f6b7007d1c0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.277+0000 7f6b78b69700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6b58060550 0x7f6b58062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.277+0000 7f6b78b69700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6b58060550 0x7f6b58062a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6b7406f2d0 tx=0x7f6b68017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.285+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b70050db0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.453 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.449+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b740ce5e0 con 0x7f6b740cf0c0 2023-10-06T02:46:08.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.449+0000 7f6b657fa700 1 -- 172.21.15.33:0/563507974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b70050940 con 0x7f6b740cf0c0 2023-10-06T02:46:08.454 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6b58060550 msgr2=0x7f6b58062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6b58060550 0x7f6b58062a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6b7406f2d0 tx=0x7f6b68017000 comp rx=0 tx=0).stop 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 msgr2=0x7f6b740cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b740cc2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000d860 tx=0x7f6b70008040 comp rx=0 tx=0).stop 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 shutdown_connections 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6b58060550 0x7f6b58062a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b740cfe10 0x7f6b74154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b740cf540 0x7f6b740c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 --2- 172.21.15.33:0/563507974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b740cf0c0 0x7f6b740cc2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 >> 172.21.15.33:0/563507974 conn(0x7f6b74066fb0 msgr2=0x7f6b74152120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 shutdown_connections 2023-10-06T02:46:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.453+0000 7f6b7a36c700 1 -- 172.21.15.33:0/563507974 wait complete. 2023-10-06T02:46:08.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:46:08.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 Processor -- start 2023-10-06T02:46:08.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 -- start start 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46d4155d20 0x7f46d40c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f46d40cb340 0x7f46d40ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d41560f0 con 0x7f46d4155d20 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d40cb7a0 con 0x7f46d40cb340 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46db811700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40c14d0 con 0x7f46d40c25b0 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46d95ad700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46d4155d20 0x7f46d40c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46d95ad700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f46d4155d20 0x7f46d40c1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54276/0 (socket says 172.21.15.33:54276) 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.573+0000 7f46d95ad700 1 -- 172.21.15.33:0/4125405435 learned_addr learned my addr 172.21.15.33:0/4125405435 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d9dae700 1 --2- 172.21.15.33:0/4125405435 >> v2:172.21.15.167:3300/0 conn(0x7f46d40cb340 0x7f46d40ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d95ad700 1 -- 172.21.15.33:0/4125405435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 msgr2=0x7f46d40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d8dac700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d95ad700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d95ad700 1 -- 172.21.15.33:0/4125405435 >> v2:172.21.15.167:3300/0 conn(0x7f46d40cb340 msgr2=0x7f46d40ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d95ad700 1 --2- 172.21.15.33:0/4125405435 >> v2:172.21.15.167:3300/0 conn(0x7f46d40cb340 0x7f46d40ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d95ad700 1 -- 172.21.15.33:0/4125405435 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40c08c0 con 0x7f46d4155d20 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d9dae700 1 --2- 172.21.15.33:0/4125405435 >> v2:172.21.15.167:3300/0 conn(0x7f46d40cb340 0x7f46d40ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d8dac700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46d95ad700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d4155d20 0x7f46d40c1960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46d40c8f00 tx=0x7f46c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5195c2c2b7f52e01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46cbfff700 1 -- 172.21.15.33:0/4125405435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c400b860 con 0x7f46d4155d20 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46cbfff700 1 -- 172.21.15.33:0/4125405435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c400b9c0 con 0x7f46d4155d20 2023-10-06T02:46:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46cbfff700 1 -- 172.21.15.33:0/4125405435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c400bd60 con 0x7f46d4155d20 2023-10-06T02:46:08.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 -- 172.21.15.33:0/4125405435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d4155d20 msgr2=0x7f46d40c1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d4155d20 0x7f46d40c1960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46d40c8f00 tx=0x7f46c4009250 comp rx=0 tx=0).stop 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 -- 172.21.15.33:0/4125405435 shutdown_connections 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 --2- 172.21.15.33:0/4125405435 >> v2:172.21.15.167:3300/0 conn(0x7f46d40cb340 0x7f46d40ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 --2- 172.21.15.33:0/4125405435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d4155d20 0x7f46d40c1960 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 -- 172.21.15.33:0/4125405435 >> 172.21.15.33:0/4125405435 conn(0x7f46d40c83f0 msgr2=0x7f46d40c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 -- 172.21.15.33:0/4125405435 shutdown_connections 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 -- 172.21.15.33:0/4125405435 wait complete. 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 Processor -- start 2023-10-06T02:46:08.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.577+0000 7f46db811700 1 -- start start 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 0x7f46d40af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d4155d20 0x7f46d40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4159d30 con 0x7f46d40cb340 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4159fa0 con 0x7f46d4155d20 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d415a280 con 0x7f46d40c25b0 2023-10-06T02:46:08.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 0x7f46d40af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 0x7f46d40af6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54288/0 (socket says 172.21.15.33:54288) 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 -- 172.21.15.33:0/1211688389 learned_addr learned my addr 172.21.15.33:0/1211688389 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d95ad700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d9dae700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d4155d20 0x7f46d40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 -- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 msgr2=0x7f46d40c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 -- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d4155d20 msgr2=0x7f46d40afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d4155d20 0x7f46d40afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 -- 172.21.15.33:0/1211688389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c40094f0 con 0x7f46d40cb340 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46d8dac700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 0x7f46d40af6f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46d000c880 tx=0x7f46d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0011600 con 0x7f46d40cb340 2023-10-06T02:46:08.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0011760 con 0x7f46d40cb340 2023-10-06T02:46:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d415a560 con 0x7f46d40cb340 2023-10-06T02:46:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0011b20 con 0x7f46d40cb340 2023-10-06T02:46:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d40c5af0 con 0x7f46d40cb340 2023-10-06T02:46:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d40710b0 con 0x7f46d40cb340 2023-10-06T02:46:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.581+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f46d00233a0 con 0x7f46d40cb340 2023-10-06T02:46:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.585+0000 7f46c97fa700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060550 0x7f46b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.585+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f46d007cf40 con 0x7f46d40cb340 2023-10-06T02:46:08.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.585+0000 7f46d95ad700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060550 0x7f46b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.585+0000 7f46d95ad700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060550 0x7f46b8062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46d40c8f00 tx=0x7f46c4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.589+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f46d0050a90 con 0x7f46d40cb340 2023-10-06T02:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.753+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f46d40c0ba0 con 0x7f46d40cb340 2023-10-06T02:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.753+0000 7f46c97fa700 1 -- 172.21.15.33:0/1211688389 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v32) v1 ==== 103+0+11 (secure 0 0 0) 0x7f46d0050620 con 0x7f46d40cb340 2023-10-06T02:46:08.758 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060550 msgr2=0x7f46b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060550 0x7f46b8062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46d40c8f00 tx=0x7f46c4017000 comp rx=0 tx=0).stop 2023-10-06T02:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 msgr2=0x7f46d40af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 0x7f46d40af6f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46d000c880 tx=0x7f46d0008040 comp rx=0 tx=0).stop 2023-10-06T02:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 shutdown_connections 2023-10-06T02:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d4155d20 0x7f46d40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d40cb340 0x7f46d40af6f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060550 0x7f46b8062a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 --2- 172.21.15.33:0/1211688389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d40c25b0 0x7f46d40c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 >> 172.21.15.33:0/1211688389 conn(0x7f46d40c83f0 msgr2=0x7f46d40c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 shutdown_connections 2023-10-06T02:46:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.757+0000 7f46db811700 1 -- 172.21.15.33:0/1211688389 wait complete. 2023-10-06T02:46:08.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.877+0000 7f7a6dfd1700 1 Processor -- start 2023-10-06T02:46:08.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- start start 2023-10-06T02:46:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a680680d0 0x7f7a680aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a68157650 0x7f7a68157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680cb6e0 con 0x7f7a68157280 2023-10-06T02:46:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680c6c10 con 0x7f7a68157650 2023-10-06T02:46:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a68158030 con 0x7f7a680680d0 2023-10-06T02:46:08.885 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a67fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a68157650 0x7f7a68157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a680680d0 0x7f7a680aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a67fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7a68157650 0x7f7a68157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54296/0 (socket says 172.21.15.33:54296) 2023-10-06T02:46:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a680680d0 0x7f7a680aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:47798/0 (socket says 172.21.15.33:47798) 2023-10-06T02:46:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a67fff700 1 -- 172.21.15.33:0/2860882468 learned_addr learned my addr 172.21.15.33:0/2860882468 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:08.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 -- 172.21.15.33:0/2860882468 >> v2:172.21.15.33:3300/0 conn(0x7f7a68157650 msgr2=0x7f7a68157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 --2- 172.21.15.33:0/2860882468 >> v2:172.21.15.33:3300/0 conn(0x7f7a68157650 0x7f7a68157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 -- 172.21.15.33:0/2860882468 >> v1:172.21.15.167:6789/0 conn(0x7f7a68157280 legacy=0x7f7a680679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 -- 172.21.15.33:0/2860882468 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680685d0 con 0x7f7a680680d0 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a66ffd700 1 --2- 172.21.15.33:0/2860882468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680aadd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a680c7210 tx=0x7f7a500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e0dc7861d41ec0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a667fc700 1 -- 172.21.15.33:0/2860882468 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50011630 con 0x7f7a680680d0 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a667fc700 1 -- 172.21.15.33:0/2860882468 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a50011790 con 0x7f7a680680d0 2023-10-06T02:46:08.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a667fc700 1 -- 172.21.15.33:0/2860882468 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50011b30 con 0x7f7a680680d0 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/2860882468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 msgr2=0x7f7a680aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/2860882468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680aadd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a680c7210 tx=0x7f7a500091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/2860882468 shutdown_connections 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/2860882468 >> v2:172.21.15.33:3300/0 conn(0x7f7a68157650 0x7f7a68157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/2860882468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680aadd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/2860882468 >> 172.21.15.33:0/2860882468 conn(0x7f7a680c83f0 msgr2=0x7f7a680c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:08.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/2860882468 shutdown_connections 2023-10-06T02:46:08.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.881+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/2860882468 wait complete. 2023-10-06T02:46:08.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.885+0000 7f7a6dfd1700 1 Processor -- start 2023-10-06T02:46:08.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.885+0000 7f7a6dfd1700 1 -- start start 2023-10-06T02:46:08.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 0x7f7a680a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a68157650 0x7f7a680acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6816a430 con 0x7f7a68157280 2023-10-06T02:46:08.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681673a0 con 0x7f7a680680d0 2023-10-06T02:46:08.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68167680 con 0x7f7a68157650 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 0x7f7a680a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 0x7f7a680a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54312/0 (socket says 172.21.15.33:54312) 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 -- 172.21.15.33:0/3711809254 learned_addr learned my addr 172.21.15.33:0/3711809254 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a677fe700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a67fff700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a68157650 0x7f7a680acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 -- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a68157650 msgr2=0x7f7a680acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a68157650 0x7f7a680acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 -- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 msgr2=0x7f7a680c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:08.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 -- 172.21.15.33:0/3711809254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009420 con 0x7f7a68157280 2023-10-06T02:46:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a66ffd700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 0x7f7a680a45a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a50013000 tx=0x7f7a50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50000d10 con 0x7f7a68157280 2023-10-06T02:46:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a50011900 con 0x7f7a68157280 2023-10-06T02:46:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68167960 con 0x7f7a68157280 2023-10-06T02:46:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50025420 con 0x7f7a68157280 2023-10-06T02:46:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.889+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a680c4450 con 0x7f7a68157280 2023-10-06T02:46:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.893+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7a50023070 con 0x7f7a68157280 2023-10-06T02:46:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.893+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6816a5a0 con 0x7f7a68157280 2023-10-06T02:46:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.893+0000 7f7a5f7fe700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7a48060550 0x7f7a48062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.893+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f7a5007e140 con 0x7f7a68157280 2023-10-06T02:46:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.893+0000 7f7a677fe700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7a48060550 0x7f7a48062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:08.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.897+0000 7f7a677fe700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7a48060550 0x7f7a48062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a58003aa0 tx=0x7f7a58000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:08.897+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7a5004e240 con 0x7f7a68157280 2023-10-06T02:46:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.061+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a680710b0 con 0x7f7a68157280 2023-10-06T02:46:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.065+0000 7f7a5f7fe700 1 -- 172.21.15.33:0/3711809254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f7a50051820 con 0x7f7a68157280 2023-10-06T02:46:09.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:05.355164+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7a48060550 msgr2=0x7f7a48062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7a48060550 0x7f7a48062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a58003aa0 tx=0x7f7a58000bc0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 msgr2=0x7f7a680a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 0x7f7a680a45a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a50013000 tx=0x7f7a50017000 comp rx=0 tx=0).stop 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 shutdown_connections 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7a48060550 0x7f7a48062a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7a68157650 0x7f7a680acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a68157280 0x7f7a680a45a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 --2- 172.21.15.33:0/3711809254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a680680d0 0x7f7a680c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 >> 172.21.15.33:0/3711809254 conn(0x7f7a680c83f0 msgr2=0x7f7a68069150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 shutdown_connections 2023-10-06T02:46:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.069+0000 7f7a6dfd1700 1 -- 172.21.15.33:0/3711809254 wait complete. 2023-10-06T02:46:09.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:46:09.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:09.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 Processor -- start 2023-10-06T02:46:09.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 -- start start 2023-10-06T02:46:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd97c156d20 0x7fd97c0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c0c8b40 0x7fd97c0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c0b07a0 con 0x7fd97c0cb500 2023-10-06T02:46:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c0b0d60 con 0x7fd97c156d20 2023-10-06T02:46:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9819cc700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c94d0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9809ca700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd97c156d20 0x7fd97c0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd973fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c0c8b40 0x7fd97c0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9809ca700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd97c156d20 0x7fd97c0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:47826/0 (socket says 172.21.15.33:47826) 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9811cb700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fd97c0cb500 0x7fd97c0cb960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47200/0 (socket says 172.21.15.33:47200) 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.197+0000 7fd9809ca700 1 -- 172.21.15.33:0/2701364439 learned_addr learned my addr 172.21.15.33:0/2701364439 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9809ca700 1 -- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c0c8b40 msgr2=0x7fd97c0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9809ca700 1 --2- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c0c8b40 0x7fd97c0c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9809ca700 1 -- 172.21.15.33:0/2701364439 >> v1:172.21.15.167:6789/0 conn(0x7fd97c0cb500 legacy=0x7fd97c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9809ca700 1 -- 172.21.15.33:0/2701364439 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c1570f0 con 0x7fd97c156d20 2023-10-06T02:46:09.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9737fe700 1 -- 172.21.15.33:0/2701364439 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529708217 0 0) 0x7fd97c0b07a0 con 0x7fd97c0cb500 2023-10-06T02:46:09.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9809ca700 1 --2- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c156d20 0x7fd97c0ce200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd978009990 tx=0x7fd978009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f25cc91f8c616f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9737fe700 1 -- 172.21.15.33:0/2701364439 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978009e80 con 0x7fd97c156d20 2023-10-06T02:46:09.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9737fe700 1 -- 172.21.15.33:0/2701364439 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd978005c10 con 0x7fd97c156d20 2023-10-06T02:46:09.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9737fe700 1 -- 172.21.15.33:0/2701364439 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800b6e0 con 0x7fd97c156d20 2023-10-06T02:46:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 -- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c156d20 msgr2=0x7fd97c0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 --2- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c156d20 0x7fd97c0ce200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd978009990 tx=0x7fd978009220 comp rx=0 tx=0).stop 2023-10-06T02:46:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 -- 172.21.15.33:0/2701364439 shutdown_connections 2023-10-06T02:46:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 --2- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c0c8b40 0x7fd97c0c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 --2- 172.21.15.33:0/2701364439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c156d20 0x7fd97c0ce200 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 -- 172.21.15.33:0/2701364439 >> 172.21.15.33:0/2701364439 conn(0x7fd97c06aab0 msgr2=0x7fd97c0cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 -- 172.21.15.33:0/2701364439 shutdown_connections 2023-10-06T02:46:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.201+0000 7fd9819cc700 1 -- 172.21.15.33:0/2701364439 wait complete. 2023-10-06T02:46:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 Processor -- start 2023-10-06T02:46:09.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 -- start start 2023-10-06T02:46:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 0x7fd97c09a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c0cb500 0x7fd97c097900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c156d20 0x7fd97c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c148910 con 0x7fd97c0c8b40 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c147d00 con 0x7fd97c0cb500 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c147fe0 con 0x7fd97c156d20 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 0x7fd97c09a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 0x7fd97c09a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54328/0 (socket says 172.21.15.33:54328) 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 -- 172.21.15.33:0/4263059217 learned_addr learned my addr 172.21.15.33:0/4263059217 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9811cb700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c156d20 0x7fd97c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd973fff700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c0cb500 0x7fd97c097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 -- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c156d20 msgr2=0x7fd97c147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c156d20 0x7fd97c147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 -- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c0cb500 msgr2=0x7fd97c097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c0cb500 0x7fd97c097900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 -- 172.21.15.33:0/4263059217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978009480 con 0x7fd97c0c8b40 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd973fff700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c0cb500 0x7fd97c097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9811cb700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c156d20 0x7fd97c147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9809ca700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 0x7fd97c09a6b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd978009260 tx=0x7fd978016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97801dcc0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c1482c0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0c33d0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97801de40 con 0x7fd97c0c8b40 2023-10-06T02:46:09.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.205+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97801a670 con 0x7fd97c0c8b40 2023-10-06T02:46:09.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.209+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c0052b0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.209+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd978011420 con 0x7fd97c0c8b40 2023-10-06T02:46:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.209+0000 7fd970ff9700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd968060550 0x7fd968062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.209+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fd97807df60 con 0x7fd97c0c8b40 2023-10-06T02:46:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.209+0000 7fd973fff700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd968060550 0x7fd968062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.209+0000 7fd973fff700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd968060550 0x7fd968062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd97c16a360 tx=0x7fd96c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.221+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd97804e0c0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.392 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd95c005ae0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd970ff9700 1 -- 172.21.15.33:0/4263059217 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9780516e0 con 0x7fd97c0c8b40 2023-10-06T02:46:09.393 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd968060550 msgr2=0x7fd968062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd968060550 0x7fd968062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd97c16a360 tx=0x7fd96c008040 comp rx=0 tx=0).stop 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 msgr2=0x7fd97c09a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 0x7fd97c09a6b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd978009260 tx=0x7fd978016000 comp rx=0 tx=0).stop 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 shutdown_connections 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd968060550 0x7fd968062a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd97c156d20 0x7fd97c147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd97c0cb500 0x7fd97c097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 --2- 172.21.15.33:0/4263059217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd97c0c8b40 0x7fd97c09a6b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 >> 172.21.15.33:0/4263059217 conn(0x7fd97c06aab0 msgr2=0x7fd97c06cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 shutdown_connections 2023-10-06T02:46:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.389+0000 7fd9819cc700 1 -- 172.21.15.33:0/4263059217 wait complete. 2023-10-06T02:46:09.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:09.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc47ce98700 1 Processor -- start 2023-10-06T02:46:09.522 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc47ce98700 1 -- start start 2023-10-06T02:46:09.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc47ce98700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc4780cf7c0 0x7fc4780b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.523 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc47ce98700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780ce580 con 0x7fc478068aa0 2023-10-06T02:46:09.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc47ce98700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780ce860 con 0x7fc478158c60 2023-10-06T02:46:09.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc47ce98700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4780cf620 con 0x7fc4780cf7c0 2023-10-06T02:46:09.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc476ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc478068aa0 0x7fc4780b29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39080/0 (socket says 172.21.15.33:39080) 2023-10-06T02:46:09.524 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.517+0000 7fc476ffd700 1 -- 172.21.15.33:0/3671408496 learned_addr learned my addr 172.21.15.33:0/3671408496 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:09.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc477fff700 1 --2- 172.21.15.33:0/3671408496 >> v2:172.21.15.167:3300/0 conn(0x7fc4780cf7c0 0x7fc4780b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc4767fc700 1 -- 172.21.15.33:0/3671408496 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708247542 0 0) 0x7fc4780ce580 con 0x7fc478068aa0 2023-10-06T02:46:09.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc4767fc700 1 -- 172.21.15.33:0/3671408496 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc460003570 con 0x7fc478068aa0 2023-10-06T02:46:09.525 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc477fff700 1 -- 172.21.15.33:0/3671408496 >> v1:172.21.15.167:6789/0 conn(0x7fc478158c60 legacy=0x7fc478068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc4767fc700 1 -- 172.21.15.33:0/3671408496 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716959983 0 0) 0x7fc460003570 con 0x7fc478068aa0 2023-10-06T02:46:09.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc477fff700 1 -- 172.21.15.33:0/3671408496 >> v1:172.21.15.33:6789/0 conn(0x7fc478068aa0 legacy=0x7fc4780b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.526 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc477fff700 1 -- 172.21.15.33:0/3671408496 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780ceb40 con 0x7fc4780cf7c0 2023-10-06T02:46:09.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc477fff700 1 --2- 172.21.15.33:0/3671408496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc4780b30f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc470009e60 tx=0x7fc470009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26098bafe169b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc4767fc700 1 -- 172.21.15.33:0/3671408496 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000e070 con 0x7fc4780cf7c0 2023-10-06T02:46:09.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc4767fc700 1 -- 172.21.15.33:0/3671408496 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000b850 con 0x7fc4780cf7c0 2023-10-06T02:46:09.527 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc4767fc700 1 -- 172.21.15.33:0/3671408496 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468001470 con 0x7fc4780cf7c0 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 -- 172.21.15.33:0/3671408496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 msgr2=0x7fc4780b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 --2- 172.21.15.33:0/3671408496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc4780b30f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc470009e60 tx=0x7fc470009780 comp rx=0 tx=0).stop 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 -- 172.21.15.33:0/3671408496 shutdown_connections 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 --2- 172.21.15.33:0/3671408496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc4780b30f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 -- 172.21.15.33:0/3671408496 >> 172.21.15.33:0/3671408496 conn(0x7fc478067a40 msgr2=0x7fc4780b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 -- 172.21.15.33:0/3671408496 shutdown_connections 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 -- 172.21.15.33:0/3671408496 wait complete. 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 Processor -- start 2023-10-06T02:46:09.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.521+0000 7fc47ce98700 1 -- start start 2023-10-06T02:46:09.530 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 0x7fc47814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc478157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478158c60 0x7fc478157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780b5b70 con 0x7fc478068aa0 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47814d430 con 0x7fc4780cf7c0 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47815b550 con 0x7fc478158c60 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 0x7fc47814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 0x7fc47814cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54332/0 (socket says 172.21.15.33:54332) 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 -- 172.21.15.33:0/564458337 learned_addr learned my addr 172.21.15.33:0/564458337 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc476ffd700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc478157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc477fff700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478158c60 0x7fc478157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 -- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478158c60 msgr2=0x7fc478157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478158c60 0x7fc478157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 -- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 msgr2=0x7fc478157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc478157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 -- 172.21.15.33:0/564458337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc470009a20 con 0x7fc478068aa0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc477fff700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478158c60 0x7fc478157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc4777fe700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 0x7fc47814cef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00c810 tx=0x7fc46c007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c012630 con 0x7fc478068aa0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c012790 con 0x7fc478068aa0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47815b830 con 0x7fc478068aa0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c012b90 con 0x7fc478068aa0 2023-10-06T02:46:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.525+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47815bdf0 con 0x7fc478068aa0 2023-10-06T02:46:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.529+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47815c010 con 0x7fc478068aa0 2023-10-06T02:46:09.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.529+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc46c01b400 con 0x7fc478068aa0 2023-10-06T02:46:09.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.529+0000 7fc45f7fe700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc464060550 0x7fc464062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.529+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fc46c07c890 con 0x7fc478068aa0 2023-10-06T02:46:09.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.529+0000 7fc476ffd700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc464060550 0x7fc464062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.529+0000 7fc476ffd700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc464060550 0x7fc464062a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc47815c9f0 tx=0x7fc468001a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.537+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc46c04c9f0 con 0x7fc478068aa0 2023-10-06T02:46:09.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc478071040 con 0x7fc478068aa0 2023-10-06T02:46:09.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc45f7fe700 1 -- 172.21.15.33:0/564458337 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc46c050010 con 0x7fc478068aa0 2023-10-06T02:46:09.705 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:09.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc464060550 msgr2=0x7fc464062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc464060550 0x7fc464062a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc47815c9f0 tx=0x7fc468001a20 comp rx=0 tx=0).stop 2023-10-06T02:46:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 msgr2=0x7fc47814cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 0x7fc47814cef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00c810 tx=0x7fc46c007ed0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 shutdown_connections 2023-10-06T02:46:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc464060550 0x7fc464062a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc478158c60 0x7fc478157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4780cf7c0 0x7fc478157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 --2- 172.21.15.33:0/564458337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc478068aa0 0x7fc47814cef0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 >> 172.21.15.33:0/564458337 conn(0x7fc478067a40 msgr2=0x7fc478068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 shutdown_connections 2023-10-06T02:46:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.701+0000 7fc47ce98700 1 -- 172.21.15.33:0/564458337 wait complete. 2023-10-06T02:46:09.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:46:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.837+0000 7f41bd0f1700 1 Processor -- start 2023-10-06T02:46:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.837+0000 7f41bd0f1700 1 -- start start 2023-10-06T02:46:09.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.837+0000 7f41bd0f1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b8156d20 0x7f41b80ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.837+0000 7f41bd0f1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80b07a0 con 0x7f41b80c8b40 2023-10-06T02:46:09.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.837+0000 7f41bd0f1700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80b0d60 con 0x7f41b80b2200 2023-10-06T02:46:09.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.837+0000 7f41bd0f1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b81570f0 con 0x7f41b8156d20 2023-10-06T02:46:09.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f41b80c8b40 0x7f41b80cb500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39084/0 (socket says 172.21.15.33:39084) 2023-10-06T02:46:09.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b77fe700 1 -- 172.21.15.33:0/39649350 learned_addr learned my addr 172.21.15.33:0/39649350 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:09.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b8156d20 0x7f41b80ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878049096 0 0) 0x7f41b80b07a0 con 0x7f41b80c8b40 2023-10-06T02:46:09.845 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a4003570 con 0x7f41b80c8b40 2023-10-06T02:46:09.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339728824 0 0) 0x7f41b80b0d60 con 0x7f41b80b2200 2023-10-06T02:46:09.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b80b07a0 con 0x7f41b80b2200 2023-10-06T02:46:09.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435231793 0 0) 0x7f41a4003570 con 0x7f41b80c8b40 2023-10-06T02:46:09.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b8156d20 msgr2=0x7f41b80ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b8156d20 0x7f41b80ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 >> v1:172.21.15.167:6789/0 conn(0x7f41b80b2200 legacy=0x7f41b80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80cca10 con 0x7f41b80c8b40 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f41a0001470 con 0x7f41b80c8b40 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 >> v1:172.21.15.33:6789/0 conn(0x7f41b80c8b40 legacy=0x7f41b80cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41a4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a4004810 0x7f41a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a40051c0 0x7f41a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b81570f0 con 0x7f41a4003c20 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400c0b0 con 0x7f41a4004810 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400c270 con 0x7f41a40051c0 2023-10-06T02:46:09.849 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b8156d20 0x7f41b80ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41a4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b77fe700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a4004810 0x7f41a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bc8f0700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a40051c0 0x7f41a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 -- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a40051c0 msgr2=0x7f41a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a40051c0 0x7f41a400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 -- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a4004810 msgr2=0x7f41a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a4004810 0x7f41a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 -- 172.21.15.33:0/39649350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4003570 con 0x7f41a4003c20 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b7fff700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41a4004010 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41b80cb450 tx=0x7f41a8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa6f16573ca1024 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a800e710 con 0x7f41a4003c20 2023-10-06T02:46:09.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a800e870 con 0x7f41a4003c20 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41b6ffd700 1 -- 172.21.15.33:0/39649350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a800ec10 con 0x7f41a4003c20 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bd0f1700 1 -- 172.21.15.33:0/39649350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 msgr2=0x7f41a4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/39649350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41a4004010 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41b80cb450 tx=0x7f41a8009120 comp rx=0 tx=0).stop 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bc8f0700 1 -- 172.21.15.33:0/39649350 reap_dead start 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bd0f1700 1 -- 172.21.15.33:0/39649350 shutdown_connections 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bd0f1700 1 -- 172.21.15.33:0/39649350 >> 172.21.15.33:0/39649350 conn(0x7f41b806aab0 msgr2=0x7f41b80cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bd0f1700 1 -- 172.21.15.33:0/39649350 shutdown_connections 2023-10-06T02:46:09.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.841+0000 7f41bd0f1700 1 -- 172.21.15.33:0/39649350 wait complete. 2023-10-06T02:46:09.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.845+0000 7f41bd0f1700 1 Processor -- start 2023-10-06T02:46:09.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.845+0000 7f41bd0f1700 1 -- start start 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41b80690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b80c1b80 0x7f41b8148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b80b1460 0x7f41b80b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80b1e10 con 0x7f41a4003c20 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80c2ce0 con 0x7f41b80b1460 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80c20d0 con 0x7f41b80c1b80 2023-10-06T02:46:09.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41b80690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41b80690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54352/0 (socket says 172.21.15.33:54352) 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 -- 172.21.15.33:0/1272032707 learned_addr learned my addr 172.21.15.33:0/1272032707 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bc8f0700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b80b1460 0x7f41b80b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b77fe700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b80c1b80 0x7f41b8148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 -- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b80c1b80 msgr2=0x7f41b8148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b80c1b80 0x7f41b8148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 -- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b80b1460 msgr2=0x7f41b80b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b80b1460 0x7f41b80b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:09.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 -- 172.21.15.33:0/1272032707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80c23b0 con 0x7f41a4003c20 2023-10-06T02:46:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41b7fff700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41b80690f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c080 tx=0x7f41a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8023070 con 0x7f41a4003c20 2023-10-06T02:46:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a800ebf0 con 0x7f41a4003c20 2023-10-06T02:46:09.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b80c2690 con 0x7f41a4003c20 2023-10-06T02:46:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a800e710 con 0x7f41a4003c20 2023-10-06T02:46:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.849+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b8069630 con 0x7f41a4003c20 2023-10-06T02:46:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.853+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b8071040 con 0x7f41a4003c20 2023-10-06T02:46:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.853+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f41a801c860 con 0x7f41a4003c20 2023-10-06T02:46:09.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.853+0000 7f41affff700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4198060550 0x7f4198062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:09.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.853+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f41a807e540 con 0x7f41a4003c20 2023-10-06T02:46:09.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.853+0000 7f41b77fe700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4198060550 0x7f4198062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:09.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.853+0000 7f41b77fe700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4198060550 0x7f4198062a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41a00015a0 tx=0x7f41a0001d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:09.861+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f41a804e600 con 0x7f41a4003c20 2023-10-06T02:46:10.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.021+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41b80ce620 con 0x7f41a4003c20 2023-10-06T02:46:10.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.021+0000 7f41affff700 1 -- 172.21.15.33:0/1272032707 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41a8017020 con 0x7f41a4003c20 2023-10-06T02:46:10.028 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4198060550 msgr2=0x7f4198062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4198060550 0x7f4198062a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41a00015a0 tx=0x7f41a0001d40 comp rx=0 tx=0).stop 2023-10-06T02:46:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 msgr2=0x7f41b80690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41b80690f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c080 tx=0x7f41a8019000 comp rx=0 tx=0).stop 2023-10-06T02:46:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 shutdown_connections 2023-10-06T02:46:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4198060550 0x7f4198062a00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41b80c1b80 0x7f41b8148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b80b1460 0x7f41b80b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 --2- 172.21.15.33:0/1272032707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41a4003c20 0x7f41b80690f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 >> 172.21.15.33:0/1272032707 conn(0x7f41b806aab0 msgr2=0x7f41b80cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 shutdown_connections 2023-10-06T02:46:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.025+0000 7f41bd0f1700 1 -- 172.21.15.33:0/1272032707 wait complete. 2023-10-06T02:46:10.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:46:10.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.149+0000 7f80affff700 1 Processor -- start 2023-10-06T02:46:10.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.149+0000 7f80affff700 1 -- start start 2023-10-06T02:46:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80b0159260 0x7f80b00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cda80 0x7f80b00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b0068100 con 0x7f80b00cd170 2023-10-06T02:46:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b00b3960 con 0x7f80b0159260 2023-10-06T02:46:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01598b0 con 0x7f80b00cda80 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80b0159260 0x7f80b00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cda80 0x7f80b00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f80b0159260 0x7f80b00cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54356/0 (socket says 172.21.15.33:54356) 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80aeffd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f80b00cd170 0x7f80b00b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:47240/0 (socket says 172.21.15.33:47240) 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 -- 172.21.15.33:0/118910048 learned_addr learned my addr 172.21.15.33:0/118910048 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 -- 172.21.15.33:0/118910048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cda80 msgr2=0x7f80b00cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80adffb700 1 -- 172.21.15.33:0/118910048 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772956654 0 0) 0x7f80b0068100 con 0x7f80b00cd170 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 --2- 172.21.15.33:0/118910048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cda80 0x7f80b00cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 -- 172.21.15.33:0/118910048 >> v1:172.21.15.167:6789/0 conn(0x7f80b00cd170 legacy=0x7f80b00b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 -- 172.21.15.33:0/118910048 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0159b90 con 0x7f80b0159260 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80ae7fc700 1 --2- 172.21.15.33:0/118910048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cd540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80a40099b0 tx=0x7f80a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb71f1355d6b9c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80adffb700 1 -- 172.21.15.33:0/118910048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4009ea0 con 0x7f80b0159260 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80adffb700 1 -- 172.21.15.33:0/118910048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a400b860 con 0x7f80b0159260 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80adffb700 1 -- 172.21.15.33:0/118910048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400bc00 con 0x7f80b0159260 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- 172.21.15.33:0/118910048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 msgr2=0x7f80b00cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 --2- 172.21.15.33:0/118910048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cd540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80a40099b0 tx=0x7f80a4009250 comp rx=0 tx=0).stop 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- 172.21.15.33:0/118910048 shutdown_connections 2023-10-06T02:46:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 --2- 172.21.15.33:0/118910048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cda80 0x7f80b00cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 --2- 172.21.15.33:0/118910048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cd540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- 172.21.15.33:0/118910048 >> 172.21.15.33:0/118910048 conn(0x7f80b006ad60 msgr2=0x7f80b006b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- 172.21.15.33:0/118910048 shutdown_connections 2023-10-06T02:46:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.153+0000 7f80affff700 1 -- 172.21.15.33:0/118910048 wait complete. 2023-10-06T02:46:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 Processor -- start 2023-10-06T02:46:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 -- start start 2023-10-06T02:46:10.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cd170 0x7f80b016d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b00cda80 0x7f80b00c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00a4c90 con 0x7f80b0159260 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00cad60 con 0x7f80b00cda80 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80affff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00cb1f0 con 0x7f80b00cd170 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54360/0 (socket says 172.21.15.33:54360) 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80aeffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cd170 0x7f80b016d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.157+0000 7f80af7fe700 1 -- 172.21.15.33:0/1283411252 learned_addr learned my addr 172.21.15.33:0/1283411252 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:10.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80ae7fc700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b00cda80 0x7f80b00c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80af7fe700 1 -- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cd170 msgr2=0x7f80b016d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80af7fe700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cd170 0x7f80b016d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80af7fe700 1 -- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b00cda80 msgr2=0x7f80b00c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80af7fe700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b00cda80 0x7f80b00c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80af7fe700 1 -- 172.21.15.33:0/1283411252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40094f0 con 0x7f80b0159260 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80af7fe700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cbdc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009e90 tx=0x7f80a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800cb10 con 0x7f80b0159260 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a800f040 con 0x7f80b0159260 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800cee0 con 0x7f80b0159260 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b00ca150 con 0x7f80b0159260 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b00ca710 con 0x7f80b0159260 2023-10-06T02:46:10.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.161+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b00caf10 con 0x7f80b0159260 2023-10-06T02:46:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.165+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f80a8014690 con 0x7f80b0159260 2023-10-06T02:46:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.165+0000 7f8096ffd700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f809c060580 0x7f809c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.165+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f80a807d620 con 0x7f80b0159260 2023-10-06T02:46:10.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.165+0000 7f80aeffd700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f809c060580 0x7f809c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.165+0000 7f80aeffd700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f809c060580 0x7f809c062a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f80a0007fd0 tx=0x7f80a0007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.169+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f80b00caf10 con 0x7f80b0159260 2023-10-06T02:46:10.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.329+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80b00caf10 con 0x7f80b0159260 2023-10-06T02:46:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.329+0000 7f8096ffd700 1 -- 172.21.15.33:0/1283411252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80b00caf10 con 0x7f80b0159260 2023-10-06T02:46:10.335 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:46:10.337 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f809c060580 msgr2=0x7f809c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f809c060580 0x7f809c062a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f80a0007fd0 tx=0x7f80a0007d50 comp rx=0 tx=0).stop 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 msgr2=0x7f80b00cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cbdc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009e90 tx=0x7f80a800b040 comp rx=0 tx=0).stop 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 shutdown_connections 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b0159260 0x7f80b00cbdc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b00cda80 0x7f80b00c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f809c060580 0x7f809c062a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 --2- 172.21.15.33:0/1283411252 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80b00cd170 0x7f80b016d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 >> 172.21.15.33:0/1283411252 conn(0x7f80b006ad60 msgr2=0x7f80b0157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:10.338 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 shutdown_connections 2023-10-06T02:46:10.339 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.333+0000 7f80affff700 1 -- 172.21.15.33:0/1283411252 wait complete. 2023-10-06T02:46:10.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:46:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.461+0000 7fa399665700 1 Processor -- start 2023-10-06T02:46:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.461+0000 7fa399665700 1 -- start start 2023-10-06T02:46:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa394158c60 0x7fa394068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa394159030 0x7fa3940688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3940ce580 con 0x7fa3940cf7c0 2023-10-06T02:46:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3940cf620 con 0x7fa394159030 2023-10-06T02:46:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3940ce860 con 0x7fa394158c60 2023-10-06T02:46:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa394159030 0x7fa3940688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa394159030 0x7fa3940688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54374/0 (socket says 172.21.15.33:54374) 2023-10-06T02:46:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 -- 172.21.15.33:0/2117332728 learned_addr learned my addr 172.21.15.33:0/2117332728 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa393fff700 1 --2- 172.21.15.33:0/2117332728 >> v2:172.21.15.167:3300/0 conn(0x7fa394158c60 0x7fa394068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 -- 172.21.15.33:0/2117332728 >> v2:172.21.15.167:3300/0 conn(0x7fa394158c60 msgr2=0x7fa394068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 --2- 172.21.15.33:0/2117332728 >> v2:172.21.15.167:3300/0 conn(0x7fa394158c60 0x7fa394068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 -- 172.21.15.33:0/2117332728 >> v1:172.21.15.167:6789/0 conn(0x7fa3940cf7c0 legacy=0x7fa39415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 -- 172.21.15.33:0/2117332728 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940ceb40 con 0x7fa394159030 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa393fff700 1 --2- 172.21.15.33:0/2117332728 >> v2:172.21.15.167:3300/0 conn(0x7fa394158c60 0x7fa394068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa398e64700 1 --2- 172.21.15.33:0/2117332728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3940688d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa39406ff30 tx=0x7fa38c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aac5ae468579dc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa392ffd700 1 -- 172.21.15.33:0/2117332728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00e070 con 0x7fa394159030 2023-10-06T02:46:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa392ffd700 1 -- 172.21.15.33:0/2117332728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c00ba10 con 0x7fa394159030 2023-10-06T02:46:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa392ffd700 1 -- 172.21.15.33:0/2117332728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0092e0 con 0x7fa394159030 2023-10-06T02:46:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- 172.21.15.33:0/2117332728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 msgr2=0x7fa3940688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 --2- 172.21.15.33:0/2117332728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3940688d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa39406ff30 tx=0x7fa38c009840 comp rx=0 tx=0).stop 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- 172.21.15.33:0/2117332728 shutdown_connections 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 --2- 172.21.15.33:0/2117332728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3940688d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 --2- 172.21.15.33:0/2117332728 >> v2:172.21.15.167:3300/0 conn(0x7fa394158c60 0x7fa394068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- 172.21.15.33:0/2117332728 >> 172.21.15.33:0/2117332728 conn(0x7fa394067a40 msgr2=0x7fa3940b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- 172.21.15.33:0/2117332728 shutdown_connections 2023-10-06T02:46:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.465+0000 7fa399665700 1 -- 172.21.15.33:0/2117332728 wait complete. 2023-10-06T02:46:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 Processor -- start 2023-10-06T02:46:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 -- start start 2023-10-06T02:46:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3940cf7c0 0x7fa39416be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa394158c60 0x7fa394156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3941567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa394156df0 con 0x7fa394159030 2023-10-06T02:46:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941570d0 con 0x7fa394158c60 2023-10-06T02:46:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa399665700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941573b0 con 0x7fa3940cf7c0 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa398e64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3941567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa398e64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3941567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54390/0 (socket says 172.21.15.33:54390) 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa3937fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa394158c60 0x7fa394156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa398e64700 1 -- 172.21.15.33:0/683957022 learned_addr learned my addr 172.21.15.33:0/683957022 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.469+0000 7fa393fff700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3940cf7c0 0x7fa39416be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa398e64700 1 -- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3940cf7c0 msgr2=0x7fa39416be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa398e64700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3940cf7c0 0x7fa39416be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa398e64700 1 -- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa394158c60 msgr2=0x7fa394156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa398e64700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa394158c60 0x7fa394156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa398e64700 1 -- 172.21.15.33:0/683957022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c009ae0 con 0x7fa394159030 2023-10-06T02:46:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa398e64700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3941567c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3940696d0 tx=0x7fa38c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0032f0 con 0x7fa394159030 2023-10-06T02:46:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c0092e0 con 0x7fa394159030 2023-10-06T02:46:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa394157690 con 0x7fa394159030 2023-10-06T02:46:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c01d420 con 0x7fa394159030 2023-10-06T02:46:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39416c3b0 con 0x7fa394159030 2023-10-06T02:46:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa394157b90 con 0x7fa394159030 2023-10-06T02:46:10.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.473+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa38c01d810 con 0x7fa394159030 2023-10-06T02:46:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.477+0000 7fa38bfff700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa374060550 0x7fa374062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.477+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fa38c07e2d0 con 0x7fa394159030 2023-10-06T02:46:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.477+0000 7fa393fff700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa374060550 0x7fa374062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.477+0000 7fa393fff700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa374060550 0x7fa374062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa39415da50 tx=0x7fa384008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.481+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa38c04e430 con 0x7fa394159030 2023-10-06T02:46:10.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.645+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa39416c5d0 con 0x7fa394159030 2023-10-06T02:46:10.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa38bfff700 1 -- 172.21.15.33:0/683957022 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v32) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa38c051a50 con 0x7fa394159030 2023-10-06T02:46:10.652 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:10.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa374060550 msgr2=0x7fa374062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa374060550 0x7fa374062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa39415da50 tx=0x7fa384008040 comp rx=0 tx=0).stop 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 msgr2=0x7fa3941567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3941567c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3940696d0 tx=0x7fa38c017000 comp rx=0 tx=0).stop 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 shutdown_connections 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa394159030 0x7fa3941567c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa394158c60 0x7fa394156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa374060550 0x7fa374062a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 --2- 172.21.15.33:0/683957022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3940cf7c0 0x7fa39416be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 >> 172.21.15.33:0/683957022 conn(0x7fa394067a40 msgr2=0x7fa394153370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 shutdown_connections 2023-10-06T02:46:10.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.649+0000 7fa399665700 1 -- 172.21.15.33:0/683957022 wait complete. 2023-10-06T02:46:10.674 INFO:tasks.cephfs_test_runner:Starting test: test_flush (tasks.cephfs.test_flush.TestFlush) 2023-10-06T02:46:10.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_flush.TestFlush.test_flush' 2023-10-06T02:46:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 Processor -- start 2023-10-06T02:46:10.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 -- start start 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f2015d340 0x7f7f200675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7f200b21e0 0x7f7f20160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f200b2600 con 0x7f7f200671e0 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f20161ba0 con 0x7f7f2015d340 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f26088700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f200b1700 con 0x7f7f200b21e0 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7f200671e0 0x7f7f2015cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39086/0 (socket says 172.21.15.33:39086) 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7f2015d340 0x7f7f200675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1f7fe700 1 -- 172.21.15.33:0/2953766814 learned_addr learned my addr 172.21.15.33:0/2953766814 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:10.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1ffff700 1 --2- 172.21.15.33:0/2953766814 >> v2:172.21.15.167:3300/0 conn(0x7f7f200b21e0 0x7f7f20160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1effd700 1 -- 172.21.15.33:0/2953766814 >> v2:172.21.15.167:3300/0 conn(0x7f7f200b21e0 msgr2=0x7f7f20160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1e7fc700 1 -- 172.21.15.33:0/2953766814 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179222425 0 0) 0x7f7f200b2600 con 0x7f7f200671e0 2023-10-06T02:46:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1effd700 1 --2- 172.21.15.33:0/2953766814 >> v2:172.21.15.167:3300/0 conn(0x7f7f200b21e0 0x7f7f20160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1effd700 1 -- 172.21.15.33:0/2953766814 >> v1:172.21.15.33:6789/0 conn(0x7f7f200671e0 legacy=0x7f7f2015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1effd700 1 -- 172.21.15.33:0/2953766814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f200b19e0 con 0x7f7f2015d340 2023-10-06T02:46:10.793 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1effd700 1 --2- 172.21.15.33:0/2953766814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f200675b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f200c6d10 tx=0x7f7f18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a827244622e307e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1e7fc700 1 -- 172.21.15.33:0/2953766814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800e070 con 0x7f7f2015d340 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1e7fc700 1 -- 172.21.15.33:0/2953766814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1800bc30 con 0x7f7f2015d340 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.785+0000 7f7f1e7fc700 1 -- 172.21.15.33:0/2953766814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18012430 con 0x7f7f2015d340 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- 172.21.15.33:0/2953766814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 msgr2=0x7f7f200675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 --2- 172.21.15.33:0/2953766814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f200675b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f200c6d10 tx=0x7f7f18009780 comp rx=0 tx=0).stop 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- 172.21.15.33:0/2953766814 shutdown_connections 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 --2- 172.21.15.33:0/2953766814 >> v2:172.21.15.167:3300/0 conn(0x7f7f200b21e0 0x7f7f20160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 --2- 172.21.15.33:0/2953766814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f200675b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- 172.21.15.33:0/2953766814 >> 172.21.15.33:0/2953766814 conn(0x7f7f200cfd70 msgr2=0x7f7f200d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- 172.21.15.33:0/2953766814 shutdown_connections 2023-10-06T02:46:10.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- 172.21.15.33:0/2953766814 wait complete. 2023-10-06T02:46:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 Processor -- start 2023-10-06T02:46:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- start start 2023-10-06T02:46:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f200671e0 0x7f7f2013ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200b21e0 0x7f7f20156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f20155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20156250 con 0x7f7f2015d340 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20157070 con 0x7f7f200b21e0 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f26088700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20153ef0 con 0x7f7f200671e0 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f20155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f1ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f20155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54404/0 (socket says 172.21.15.33:54404) 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f1ffff700 1 -- 172.21.15.33:0/1391214759 learned_addr learned my addr 172.21.15.33:0/1391214759 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f1f7fe700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f200671e0 0x7f7f2013ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.789+0000 7f7f1effd700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200b21e0 0x7f7f20156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1ffff700 1 -- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f200671e0 msgr2=0x7f7f2013ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1ffff700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f200671e0 0x7f7f2013ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1ffff700 1 -- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200b21e0 msgr2=0x7f7f20156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1ffff700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200b21e0 0x7f7f20156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1ffff700 1 -- 172.21.15.33:0/1391214759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f18009a20 con 0x7f7f2015d340 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1effd700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200b21e0 0x7f7f20156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f1ffff700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f20155320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000d8c0 tx=0x7f7f10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100125f0 con 0x7f7f2015d340 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10012770 con 0x7f7f2015d340 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f20155c90 con 0x7f7f2015d340 2023-10-06T02:46:10.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10012b30 con 0x7f7f2015d340 2023-10-06T02:46:10.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f20153b80 con 0x7f7f2015d340 2023-10-06T02:46:10.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.793+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f20154080 con 0x7f7f2015d340 2023-10-06T02:46:10.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.797+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7f1001e460 con 0x7f7f2015d340 2023-10-06T02:46:10.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.797+0000 7f7f177fe700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7f00060580 0x7f7f00062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:10.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.797+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f7f1007d0e0 con 0x7f7f2015d340 2023-10-06T02:46:10.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.797+0000 7f7f1f7fe700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7f00060580 0x7f7f00062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:10.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.797+0000 7f7f1f7fe700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7f00060580 0x7f7f00062a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f08007160 tx=0x7f7f08007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:10.809 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:10.805+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7f1004d240 con 0x7f7f2015d340 2023-10-06T02:46:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.021+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_flush.TestFlush.test_flush"]} v 0) v1 -- 0x7f7f200710b0 con 0x7f7f2015d340 2023-10-06T02:46:11.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.397+0000 7f7f177fe700 1 -- 172.21.15.33:0/1391214759 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_flush.TestFlush.test_flush"]}]=0 v65) v1 ==== 126+0+0 (secure 0 0 0) 0x7f7f10050860 con 0x7f7f2015d340 2023-10-06T02:46:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7f00060580 msgr2=0x7f7f00062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7f00060580 0x7f7f00062a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f08007160 tx=0x7f7f08007f50 comp rx=0 tx=0).stop 2023-10-06T02:46:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 msgr2=0x7f7f20155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f20155320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000d8c0 tx=0x7f7f10008040 comp rx=0 tx=0).stop 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 shutdown_connections 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f2015d340 0x7f7f20155320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f200b21e0 0x7f7f20156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7f00060580 0x7f7f00062a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 --2- 172.21.15.33:0/1391214759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f200671e0 0x7f7f2013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 >> 172.21.15.33:0/1391214759 conn(0x7f7f200cfd70 msgr2=0x7f7f2015c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 shutdown_connections 2023-10-06T02:46:11.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.401+0000 7f7f26088700 1 -- 172.21.15.33:0/1391214759 wait complete. 2023-10-06T02:46:11.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T02:46:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.537+0000 7f38d0d8b700 1 Processor -- start 2023-10-06T02:46:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.537+0000 7f38d0d8b700 1 -- start start 2023-10-06T02:46:11.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38cc158c60 0x7f38cc159030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc0cb1f0 0x7f38cc15cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0cd7f0 con 0x7f38cc15cb70 2023-10-06T02:46:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0cddb0 con 0x7f38cc158c60 2023-10-06T02:46:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc15d630 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38cc158c60 0x7f38cc159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38caffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f38cc15cb70 0x7f38cc0caae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39092/0 (socket says 172.21.15.33:39092) 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38caffd700 1 -- 172.21.15.33:0/1338313749 learned_addr learned my addr 172.21.15.33:0/1338313749 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cbfff700 1 --2- 172.21.15.33:0/1338313749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc0cb1f0 0x7f38cc15cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cb7fe700 1 -- 172.21.15.33:0/1338313749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc0cb1f0 msgr2=0x7f38cc15cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cb7fe700 1 --2- 172.21.15.33:0/1338313749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc0cb1f0 0x7f38cc15cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cb7fe700 1 -- 172.21.15.33:0/1338313749 >> v1:172.21.15.33:6789/0 conn(0x7f38cc15cb70 legacy=0x7f38cc0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cb7fe700 1 -- 172.21.15.33:0/1338313749 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc15d910 con 0x7f38cc158c60 2023-10-06T02:46:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38ca7fc700 1 -- 172.21.15.33:0/1338313749 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241817004 0 0) 0x7f38cc0cd7f0 con 0x7f38cc15cb70 2023-10-06T02:46:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38cb7fe700 1 --2- 172.21.15.33:0/1338313749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc158c60 0x7f38cc159030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0098e0 tx=0x7f38bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260ba9c5773fcdea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38ca7fc700 1 -- 172.21.15.33:0/1338313749 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00b640 con 0x7f38cc158c60 2023-10-06T02:46:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38ca7fc700 1 -- 172.21.15.33:0/1338313749 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc00b7a0 con 0x7f38cc158c60 2023-10-06T02:46:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38ca7fc700 1 -- 172.21.15.33:0/1338313749 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00bb40 con 0x7f38cc158c60 2023-10-06T02:46:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- 172.21.15.33:0/1338313749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc158c60 msgr2=0x7f38cc159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/1338313749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc158c60 0x7f38cc159030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0098e0 tx=0x7f38bc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- 172.21.15.33:0/1338313749 shutdown_connections 2023-10-06T02:46:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/1338313749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc0cb1f0 0x7f38cc15cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/1338313749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc158c60 0x7f38cc159030 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- 172.21.15.33:0/1338313749 >> 172.21.15.33:0/1338313749 conn(0x7f38cc067a80 msgr2=0x7f38cc0cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- 172.21.15.33:0/1338313749 shutdown_connections 2023-10-06T02:46:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.541+0000 7f38d0d8b700 1 -- 172.21.15.33:0/1338313749 wait complete. 2023-10-06T02:46:11.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 Processor -- start 2023-10-06T02:46:11.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 -- start start 2023-10-06T02:46:11.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 0x7f38cc14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:11.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc158c60 0x7f38cc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:11.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc15cb70 0x7f38cc15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:11.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc150150 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc150430 con 0x7f38cc158c60 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38d0d8b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc150710 con 0x7f38cc15cb70 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.545+0000 7f38cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 0x7f38cc14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 0x7f38cc14eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54418/0 (socket says 172.21.15.33:54418) 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 -- 172.21.15.33:0/2002986598 learned_addr learned my addr 172.21.15.33:0/2002986598 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cbfff700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc15cb70 0x7f38cc15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38caffd700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc158c60 0x7f38cc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 -- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc15cb70 msgr2=0x7f38cc15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc15cb70 0x7f38cc15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 -- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc158c60 msgr2=0x7f38cc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc158c60 0x7f38cc15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 -- 172.21.15.33:0/2002986598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc009420 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cbfff700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc15cb70 0x7f38cc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38cb7fe700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 0x7f38cc14eac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0696e0 tx=0x7f38bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00bb20 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc000c90 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc1509f0 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc011630 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc14f000 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.549+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc071040 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.553+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f38bc0119a0 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.553+0000 7f38b37fe700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f38b8060550 0x7f38b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.553+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f38bc07dff0 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.553+0000 7f38caffd700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f38b8060550 0x7f38b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.553+0000 7f38caffd700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f38b8060550 0x7f38b8062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38cc152f10 tx=0x7f38c0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.557+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f38bc0adbe0 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.717+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38cc0cda40 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.721+0000 7f38b37fe700 1 -- 172.21.15.33:0/2002986598 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38bc04e0d0 con 0x7f38cc0cb1f0 2023-10-06T02:46:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f38b8060550 msgr2=0x7f38b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f38b8060550 0x7f38b8062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38cc152f10 tx=0x7f38c0007d30 comp rx=0 tx=0).stop 2023-10-06T02:46:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 msgr2=0x7f38cc14eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 0x7f38cc14eac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0696e0 tx=0x7f38bc017000 comp rx=0 tx=0).stop 2023-10-06T02:46:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 shutdown_connections 2023-10-06T02:46:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f38b8060550 0x7f38b8062a00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f38cc15cb70 0x7f38cc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38cc158c60 0x7f38cc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 --2- 172.21.15.33:0/2002986598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38cc0cb1f0 0x7f38cc14eac0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 >> 172.21.15.33:0/2002986598 conn(0x7f38cc067a80 msgr2=0x7f38cc0ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 shutdown_connections 2023-10-06T02:46:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:11.725+0000 7f38d0d8b700 1 -- 172.21.15.33:0/2002986598 wait complete. 2023-10-06T02:46:11.742 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:46:11.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:46:11.761 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:46:11.782 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:46:11.783 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T02:46:11.783 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:46:11.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:11.798 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T02:46:11.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:46:11.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:11.799 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:46:11.847 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi167.front.sepia.ceph.com... 2023-10-06T02:46:11.847 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:46:11.847 DEBUG:teuthology.orchestra.run.smithi167:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T02:46:11.885 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T02:46:11.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:46:11.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:11.885 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:46:11.946 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T02:46:11.947 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:46:11.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-06T02:46:11.964 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T02:46:11.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:46:11.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:11.966 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T02:46:11.973 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi167.front.sepia.ceph.com... 2023-10-06T02:46:11.973 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:46:11.973 DEBUG:teuthology.orchestra.run.smithi167:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-06T02:46:11.988 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T02:46:11.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:46:11.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:11.991 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T02:46:12.039 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T02:46:12.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 Processor -- start 2023-10-06T02:46:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- start start 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fd40ac730 0x7f3fd406bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd40c7e80 con 0x7f3fd406c500 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd406c980 con 0x7f3fd40ac730 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40acba0 con 0x7f3fd4156940 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fd40ac730 0x7f3fd406bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fd40ac730 0x7f3fd406bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54422/0 (socket says 172.21.15.33:54422) 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 -- 172.21.15.33:0/200731937 learned_addr learned my addr 172.21.15.33:0/200731937 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd2d9d700 1 --2- 172.21.15.33:0/200731937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 -- 172.21.15.33:0/200731937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 msgr2=0x7f3fd4156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 --2- 172.21.15.33:0/200731937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd4156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 -- 172.21.15.33:0/200731937 >> v1:172.21.15.167:6789/0 conn(0x7f3fd406c500 legacy=0x7f3fd4067f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 -- 172.21.15.33:0/200731937 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd40c7270 con 0x7f3fd40ac730 2023-10-06T02:46:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fca59c700 1 --2- 172.21.15.33:0/200731937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd406bfc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099b0 tx=0x7f3fbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7fca9e95a1046f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd259c700 1 -- 172.21.15.33:0/200731937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc009ea0 con 0x7f3fd40ac730 2023-10-06T02:46:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd259c700 1 -- 172.21.15.33:0/200731937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00b860 con 0x7f3fd40ac730 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd259c700 1 -- 172.21.15.33:0/200731937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00bc00 con 0x7f3fd40ac730 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- 172.21.15.33:0/200731937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 msgr2=0x7f3fd406bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 --2- 172.21.15.33:0/200731937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd406bfc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099b0 tx=0x7f3fbc009250 comp rx=0 tx=0).stop 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- 172.21.15.33:0/200731937 shutdown_connections 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 --2- 172.21.15.33:0/200731937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd406bfc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 --2- 172.21.15.33:0/200731937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- 172.21.15.33:0/200731937 >> 172.21.15.33:0/200731937 conn(0x7f3fd406e7c0 msgr2=0x7f3fd406ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- 172.21.15.33:0/200731937 shutdown_connections 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.157+0000 7f3fd9213700 1 -- 172.21.15.33:0/200731937 wait complete. 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 Processor -- start 2023-10-06T02:46:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 -- start start 2023-10-06T02:46:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fd406c500 0x7f3fd413e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd40ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd40af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd413e9f0 con 0x7f3fd40ac730 2023-10-06T02:46:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40ada40 con 0x7f3fd406c500 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40add20 con 0x7f3fd4156940 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd40ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd40ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54430/0 (socket says 172.21.15.33:54430) 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 -- 172.21.15.33:0/3343741833 learned_addr learned my addr 172.21.15.33:0/3343741833 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd359e700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd2d9d700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fd406c500 0x7f3fd413e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 -- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 msgr2=0x7f3fd40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd40af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 -- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fd406c500 msgr2=0x7f3fd413e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fd406c500 0x7f3fd413e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 -- 172.21.15.33:0/3343741833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0094f0 con 0x7f3fd40ac730 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fca59c700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd40ad010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40cc960 tx=0x7f3fbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc000d10 con 0x7f3fd40ac730 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00bbe0 con 0x7f3fd40ac730 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011670 con 0x7f3fd40ac730 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd40ae000 con 0x7f3fd40ac730 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.161+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd40ae5c0 con 0x7f3fd40ac730 2023-10-06T02:46:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.165+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd40710b0 con 0x7f3fd40ac730 2023-10-06T02:46:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.165+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3fbc011aa0 con 0x7f3fd40ac730 2023-10-06T02:46:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.165+0000 7f3fcb7fe700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fb4060550 0x7f3fb4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.165+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f3fbc07e4b0 con 0x7f3fd40ac730 2023-10-06T02:46:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.165+0000 7f3fd2d9d700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fb4060550 0x7f3fb4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.165+0000 7f3fd2d9d700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fb4060550 0x7f3fb4062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009d10 tx=0x7f3fc40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.173+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3fbc04e570 con 0x7f3fd40ac730 2023-10-06T02:46:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.373+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd40c7550 con 0x7f3fd40ac730 2023-10-06T02:46:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.373+0000 7f3fcb7fe700 1 -- 172.21.15.33:0/3343741833 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f3fbc051b90 con 0x7f3fd40ac730 2023-10-06T02:46:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:12.378 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:46:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fb4060550 msgr2=0x7f3fb4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fb4060550 0x7f3fb4062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009d10 tx=0x7f3fc40091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 msgr2=0x7f3fd40ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd40ad010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40cc960 tx=0x7f3fbc017000 comp rx=0 tx=0).stop 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 shutdown_connections 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fb4060550 0x7f3fb4062a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fd4156940 0x7f3fd40af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd40ac730 0x7f3fd40ad010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 --2- 172.21.15.33:0/3343741833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fd406c500 0x7f3fd413e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 >> 172.21.15.33:0/3343741833 conn(0x7f3fd406e7c0 msgr2=0x7f3fd406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 shutdown_connections 2023-10-06T02:46:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.377+0000 7f3fd9213700 1 -- 172.21.15.33:0/3343741833 wait complete. 2023-10-06T02:46:12.382 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:46:12.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 Processor -- start 2023-10-06T02:46:12.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 -- start start 2023-10-06T02:46:12.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd560156b10 0x7fd560157b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd5600cbe10 0x7fd5600cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56015b620 con 0x7fd5600ca070 2023-10-06T02:46:12.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600cc7a0 con 0x7fd560156b10 2023-10-06T02:46:12.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd5656be700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5601580c0 con 0x7fd5600cbe10 2023-10-06T02:46:12.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd560156b10 0x7fd560157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd564ebd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd5600ca070 0x7fd5600ca4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39098/0 (socket says 172.21.15.33:39098) 2023-10-06T02:46:12.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd564ebd700 1 -- 172.21.15.33:0/3301975658 learned_addr learned my addr 172.21.15.33:0/3301975658 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:12.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55f7fe700 1 --2- 172.21.15.33:0/3301975658 >> v2:172.21.15.167:3300/0 conn(0x7fd5600cbe10 0x7fd5600cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55ffff700 1 -- 172.21.15.33:0/3301975658 >> v2:172.21.15.167:3300/0 conn(0x7fd5600cbe10 msgr2=0x7fd5600cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:12.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55ffff700 1 --2- 172.21.15.33:0/3301975658 >> v2:172.21.15.167:3300/0 conn(0x7fd5600cbe10 0x7fd5600cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55ffff700 1 -- 172.21.15.33:0/3301975658 >> v1:172.21.15.33:6789/0 conn(0x7fd5600ca070 legacy=0x7fd5600ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55ffff700 1 -- 172.21.15.33:0/3301975658 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56015a730 con 0x7fd560156b10 2023-10-06T02:46:12.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55f7fe700 1 --2- 172.21.15.33:0/3301975658 >> v2:172.21.15.167:3300/0 conn(0x7fd5600cbe10 0x7fd5600cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:12.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55ffff700 1 --2- 172.21.15.33:0/3301975658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd560157b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd56006f2f0 tx=0x7fd5500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729fa6f8d49bbbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55effd700 1 -- 172.21.15.33:0/3301975658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000b6b0 con 0x7fd560156b10 2023-10-06T02:46:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55effd700 1 -- 172.21.15.33:0/3301975658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000b810 con 0x7fd560156b10 2023-10-06T02:46:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.497+0000 7fd55effd700 1 -- 172.21.15.33:0/3301975658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000bbb0 con 0x7fd560156b10 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 -- 172.21.15.33:0/3301975658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 msgr2=0x7fd560157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 --2- 172.21.15.33:0/3301975658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd560157b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd56006f2f0 tx=0x7fd5500091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 -- 172.21.15.33:0/3301975658 shutdown_connections 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 --2- 172.21.15.33:0/3301975658 >> v2:172.21.15.167:3300/0 conn(0x7fd5600cbe10 0x7fd5600cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 --2- 172.21.15.33:0/3301975658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd560157b80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 -- 172.21.15.33:0/3301975658 >> 172.21.15.33:0/3301975658 conn(0x7fd560086ff0 msgr2=0x7fd560096cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 -- 172.21.15.33:0/3301975658 shutdown_connections 2023-10-06T02:46:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.501+0000 7fd5656be700 1 -- 172.21.15.33:0/3301975658 wait complete. 2023-10-06T02:46:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 Processor -- start 2023-10-06T02:46:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- start start 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5600ca070 0x7fd5600c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5600cbe10 0x7fd560150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd5600ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600c5c10 con 0x7fd560156b10 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600c5240 con 0x7fd5600ca070 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600accb0 con 0x7fd5600cbe10 2023-10-06T02:46:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd5600ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd5600ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54450/0 (socket says 172.21.15.33:54450) 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 -- 172.21.15.33:0/3600107899 learned_addr learned my addr 172.21.15.33:0/3600107899 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd55f7fe700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5600cbe10 0x7fd560150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd55ffff700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5600ca070 0x7fd5600c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 -- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5600cbe10 msgr2=0x7fd560150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5600cbe10 0x7fd560150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 -- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5600ca070 msgr2=0x7fd5600c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5600ca070 0x7fd5600c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 -- 172.21.15.33:0/3600107899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500093c0 con 0x7fd560156b10 2023-10-06T02:46:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd564ebd700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd5600ac3b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd55800cbc0 tx=0x7fd558001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800a8e0 con 0x7fd560156b10 2023-10-06T02:46:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5600acf90 con 0x7fd560156b10 2023-10-06T02:46:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5600c4ca0 con 0x7fd560156b10 2023-10-06T02:46:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800ac50 con 0x7fd560156b10 2023-10-06T02:46:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800adc0 con 0x7fd560156b10 2023-10-06T02:46:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.505+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5380052b0 con 0x7fd560156b10 2023-10-06T02:46:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.509+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd5580070f0 con 0x7fd560156b10 2023-10-06T02:46:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.509+0000 7fd557fff700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd544060550 0x7fd544062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.509+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fd55807ccb0 con 0x7fd560156b10 2023-10-06T02:46:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.509+0000 7fd55ffff700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd544060550 0x7fd544062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.509+0000 7fd55ffff700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd544060550 0x7fd544062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd550009390 tx=0x7fd550017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.513+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd55804ce90 con 0x7fd560156b10 2023-10-06T02:46:12.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.677+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd538005830 con 0x7fd560156b10 2023-10-06T02:46:12.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.677+0000 7fd557fff700 1 -- 172.21.15.33:0/3600107899 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fd5580504b0 con 0x7fd560156b10 2023-10-06T02:46:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:12.684 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:05.355164+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd544060550 msgr2=0x7fd544062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd544060550 0x7fd544062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd550009390 tx=0x7fd550017000 comp rx=0 tx=0).stop 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 msgr2=0x7fd5600ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd5600ac3b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd55800cbc0 tx=0x7fd558001970 comp rx=0 tx=0).stop 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 shutdown_connections 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd544060550 0x7fd544062a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5600cbe10 0x7fd560150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd560156b10 0x7fd5600ac3b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 --2- 172.21.15.33:0/3600107899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5600ca070 0x7fd5600c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 >> 172.21.15.33:0/3600107899 conn(0x7fd560086ff0 msgr2=0x7fd560155850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 shutdown_connections 2023-10-06T02:46:12.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.681+0000 7fd5656be700 1 -- 172.21.15.33:0/3600107899 wait complete. 2023-10-06T02:46:12.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:45:21.485516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:45:22.142461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-06T02:45:25.344940+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:46:12.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 Processor -- start 2023-10-06T02:46:12.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 -- start start 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f780c0c7e50 0x7f780c066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f780c155c80 0x7f780c0674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c0c5e30 con 0x7f780c1558b0 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0c69b0 con 0x7f780c0c7e50 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7811b20700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0c6080 con 0x7f780c155c80 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f780c0c7e50 0x7f780c066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f780c0c7e50 0x7f780c066fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53188/0 (socket says 172.21.15.33:53188) 2023-10-06T02:46:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 -- 172.21.15.33:0/2619082452 learned_addr learned my addr 172.21.15.33:0/2619082452 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f781131f700 1 --2- 172.21.15.33:0/2619082452 >> v2:172.21.15.167:3300/0 conn(0x7f780c155c80 0x7f780c0674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 -- 172.21.15.33:0/2619082452 >> v2:172.21.15.167:3300/0 conn(0x7f780c155c80 msgr2=0x7f780c0674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 --2- 172.21.15.33:0/2619082452 >> v2:172.21.15.167:3300/0 conn(0x7f780c155c80 0x7f780c0674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 -- 172.21.15.33:0/2619082452 >> v1:172.21.15.167:6789/0 conn(0x7f780c1558b0 legacy=0x7f780c0ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 -- 172.21.15.33:0/2619082452 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0c6360 con 0x7f780c0c7e50 2023-10-06T02:46:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f781131f700 1 --2- 172.21.15.33:0/2619082452 >> v2:172.21.15.167:3300/0 conn(0x7f780c155c80 0x7f780c0674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.809+0000 7f7803fff700 1 --2- 172.21.15.33:0/2619082452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0c7e50 0x7f780c066fb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f780c0c5dc0 tx=0x7f77fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf9a6176c2b69fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f78037fe700 1 -- 172.21.15.33:0/2619082452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc009e70 con 0x7f780c0c7e50 2023-10-06T02:46:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f78037fe700 1 -- 172.21.15.33:0/2619082452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc00b860 con 0x7f780c0c7e50 2023-10-06T02:46:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f78037fe700 1 -- 172.21.15.33:0/2619082452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00bc00 con 0x7f780c0c7e50 2023-10-06T02:46:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 -- 172.21.15.33:0/2619082452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0c7e50 msgr2=0x7f780c066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 --2- 172.21.15.33:0/2619082452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0c7e50 0x7f780c066fb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f780c0c5dc0 tx=0x7f77fc009250 comp rx=0 tx=0).stop 2023-10-06T02:46:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 -- 172.21.15.33:0/2619082452 shutdown_connections 2023-10-06T02:46:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 --2- 172.21.15.33:0/2619082452 >> v2:172.21.15.167:3300/0 conn(0x7f780c155c80 0x7f780c0674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 --2- 172.21.15.33:0/2619082452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c0c7e50 0x7f780c066fb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 -- 172.21.15.33:0/2619082452 >> 172.21.15.33:0/2619082452 conn(0x7f780c0cc2c0 msgr2=0x7f780c0c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 -- 172.21.15.33:0/2619082452 shutdown_connections 2023-10-06T02:46:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.813+0000 7f7811b20700 1 -- 172.21.15.33:0/2619082452 wait complete. 2023-10-06T02:46:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 Processor -- start 2023-10-06T02:46:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 -- start start 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c1558b0 0x7f780c0c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 0x7f780c0c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c0aec50 0x7f780c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0bf940 con 0x7f780c155c80 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1451a0 con 0x7f780c1558b0 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7811b20700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0c1ef0 con 0x7f780c0aec50 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 0x7f780c0c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7810b1e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c1558b0 0x7f780c0c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 0x7f780c0c1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53198/0 (socket says 172.21.15.33:53198) 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f781131f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c0aec50 0x7f780c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 -- 172.21.15.33:0/1523812298 learned_addr learned my addr 172.21.15.33:0/1523812298 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 -- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c0aec50 msgr2=0x7f780c148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c0aec50 0x7f780c148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 -- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c1558b0 msgr2=0x7f780c0c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c1558b0 0x7f780c0c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.817+0000 7f7803fff700 1 -- 172.21.15.33:0/1523812298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7804009a20 con 0x7f780c155c80 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7803fff700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 0x7f780c0c1630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f780c06c9a0 tx=0x7f77fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00b9d0 con 0x7f780c155c80 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc00bb30 con 0x7f780c155c80 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc0094f0 con 0x7f780c155c80 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc025670 con 0x7f780c155c80 2023-10-06T02:46:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c0bf2f0 con 0x7f780c155c80 2023-10-06T02:46:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c0bfab0 con 0x7f780c155c80 2023-10-06T02:46:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f77fc023070 con 0x7f780c155c80 2023-10-06T02:46:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7800ff9700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f77f8060580 0x7f77f8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f77fc07e030 con 0x7f780c155c80 2023-10-06T02:46:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.821+0000 7f7810b1e700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f77f8060580 0x7f77f8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.825+0000 7f7810b1e700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f77f8060580 0x7f77f8062a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f780c0aebe0 tx=0x7f7808009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.829+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f77fc04e250 con 0x7f780c155c80 2023-10-06T02:46:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.993+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f780c071040 con 0x7f780c155c80 2023-10-06T02:46:12.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.993+0000 7f7800ff9700 1 -- 172.21.15.33:0/1523812298 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77fc051870 con 0x7f780c155c80 2023-10-06T02:46:12.999 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:13.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f77f8060580 msgr2=0x7f77f8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f77f8060580 0x7f77f8062a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f780c0aebe0 tx=0x7f7808009040 comp rx=0 tx=0).stop 2023-10-06T02:46:13.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 msgr2=0x7f780c0c1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 0x7f780c0c1630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f780c06c9a0 tx=0x7f77fc017000 comp rx=0 tx=0).stop 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 shutdown_connections 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f77f8060580 0x7f77f8062a30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f780c0aec50 0x7f780c148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f780c155c80 0x7f780c0c1630 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 --2- 172.21.15.33:0/1523812298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f780c1558b0 0x7f780c0c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 >> 172.21.15.33:0/1523812298 conn(0x7f780c0cc2c0 msgr2=0x7f780c14e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 shutdown_connections 2023-10-06T02:46:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:12.997+0000 7f7811b20700 1 -- 172.21.15.33:0/1523812298 wait complete. 2023-10-06T02:46:13.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:13.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43db735700 1 Processor -- start 2023-10-06T02:46:13.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43db735700 1 -- start start 2023-10-06T02:46:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43db735700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43db735700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d415d830 con 0x7f43d4158bf0 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43db735700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d40b00a0 con 0x7f43d40b0530 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43db735700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4067c10 con 0x7f43d4158fc0 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43d94d1700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43d4158bf0 0x7f43d40b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60422/0 (socket says 172.21.15.33:60422) 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43d94d1700 1 -- 172.21.15.33:0/2470461101 learned_addr learned my addr 172.21.15.33:0/2470461101 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43d9cd2700 1 --2- 172.21.15.33:0/2470461101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115622037 0 0) 0x7f43d415d830 con 0x7f43d4158bf0 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c0003570 con 0x7f43d4158bf0 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092526320 0 0) 0x7f43d40b00a0 con 0x7f43d40b0530 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d415d830 con 0x7f43d40b0530 2023-10-06T02:46:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43d9cd2700 1 -- 172.21.15.33:0/2470461101 >> v1:172.21.15.167:6789/0 conn(0x7f43d40b0530 legacy=0x7f43d415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43d9cd2700 1 -- 172.21.15.33:0/2470461101 >> v1:172.21.15.33:6789/0 conn(0x7f43d4158bf0 legacy=0x7f43d40b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.125+0000 7f43d9cd2700 1 -- 172.21.15.33:0/2470461101 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40cba20 con 0x7f43d4158fc0 2023-10-06T02:46:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43d9cd2700 1 --2- 172.21.15.33:0/2470461101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415d180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009e90 tx=0x7f43cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756c29084a44f6a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00e070 con 0x7f43d4158fc0 2023-10-06T02:46:13.134 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc00bb80 con 0x7f43d4158fc0 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43cbfff700 1 -- 172.21.15.33:0/2470461101 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc012430 con 0x7f43d4158fc0 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- 172.21.15.33:0/2470461101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 msgr2=0x7f43d415d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 --2- 172.21.15.33:0/2470461101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415d180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009e90 tx=0x7f43cc009780 comp rx=0 tx=0).stop 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- 172.21.15.33:0/2470461101 shutdown_connections 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 --2- 172.21.15.33:0/2470461101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415d180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- 172.21.15.33:0/2470461101 >> 172.21.15.33:0/2470461101 conn(0x7f43d40cfd70 msgr2=0x7f43d40d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- 172.21.15.33:0/2470461101 shutdown_connections 2023-10-06T02:46:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- 172.21.15.33:0/2470461101 wait complete. 2023-10-06T02:46:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 Processor -- start 2023-10-06T02:46:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- start start 2023-10-06T02:46:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 0x7f43d4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d415c050 0x7f43d415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415b680 con 0x7f43d4158bf0 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41561b0 con 0x7f43d415c050 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.129+0000 7f43db735700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415aa70 con 0x7f43d4158fc0 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 0x7f43d4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 0x7f43d4155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53208/0 (socket says 172.21.15.33:53208) 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 -- 172.21.15.33:0/4128510265 learned_addr learned my addr 172.21.15.33:0/4128510265 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d9cd2700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d415c050 0x7f43d415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d8cd0700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 -- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 msgr2=0x7f43d415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 -- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d415c050 msgr2=0x7f43d415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d415c050 0x7f43d415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 -- 172.21.15.33:0/4128510265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009a20 con 0x7f43d4158bf0 2023-10-06T02:46:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d94d1700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 0x7f43d4155c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c790 tx=0x7f43d0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0005c00 con 0x7f43d4158bf0 2023-10-06T02:46:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d0005d60 con 0x7f43d4158bf0 2023-10-06T02:46:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d415ad50 con 0x7f43d4158bf0 2023-10-06T02:46:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00072e0 con 0x7f43d4158bf0 2023-10-06T02:46:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d415b310 con 0x7f43d4158bf0 2023-10-06T02:46:13.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d40b14f0 con 0x7f43d4158bf0 2023-10-06T02:46:13.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f43d001b3b0 con 0x7f43d4158bf0 2023-10-06T02:46:13.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43c97fa700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f43b8060550 0x7f43b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f43d007c160 con 0x7f43d4158bf0 2023-10-06T02:46:13.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.133+0000 7f43d8cd0700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f43b8060550 0x7f43b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.137+0000 7f43d8cd0700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f43b8060550 0x7f43b8062a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43d4159fb0 tx=0x7f43c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.141+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f43d40b14f0 con 0x7f43d4158bf0 2023-10-06T02:46:13.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.305+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43d40b14f0 con 0x7f43d4158bf0 2023-10-06T02:46:13.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43c97fa700 1 -- 172.21.15.33:0/4128510265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f43d40b14f0 con 0x7f43d4158bf0 2023-10-06T02:46:13.312 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f43b8060550 msgr2=0x7f43b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f43b8060550 0x7f43b8062a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43d4159fb0 tx=0x7f43c4009040 comp rx=0 tx=0).stop 2023-10-06T02:46:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 msgr2=0x7f43d4155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 0x7f43d4155c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c790 tx=0x7f43d0007ea0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 shutdown_connections 2023-10-06T02:46:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f43b8060550 0x7f43b8062a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43d4158fc0 0x7f43d415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43d415c050 0x7f43d415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 --2- 172.21.15.33:0/4128510265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43d4158bf0 0x7f43d4155c70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 >> 172.21.15.33:0/4128510265 conn(0x7f43d40cfd70 msgr2=0x7f43d415cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 shutdown_connections 2023-10-06T02:46:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.309+0000 7f43db735700 1 -- 172.21.15.33:0/4128510265 wait complete. 2023-10-06T02:46:13.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:46:13.439 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.433+0000 7f7d3b324700 1 Processor -- start 2023-10-06T02:46:13.440 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.433+0000 7f7d3b324700 1 -- start start 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d34158c60 0x7f7d34068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7d340cf7c0 0x7f7d340688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d3415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d340cf620 con 0x7f7d34158c60 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d340ce580 con 0x7f7d340cf7c0 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340ce860 con 0x7f7d34159030 2023-10-06T02:46:13.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d34158c60 0x7f7d34068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d34158c60 0x7f7d34068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53214/0 (socket says 172.21.15.33:53214) 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 -- 172.21.15.33:0/3015507756 learned_addr learned my addr 172.21.15.33:0/3015507756 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d39b21700 1 --2- 172.21.15.33:0/3015507756 >> v2:172.21.15.167:3300/0 conn(0x7f7d340cf7c0 0x7f7d340688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3ab23700 1 --2- 172.21.15.33:0/3015507756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d3415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 -- 172.21.15.33:0/3015507756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 msgr2=0x7f7d3415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 --2- 172.21.15.33:0/3015507756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d3415ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 -- 172.21.15.33:0/3015507756 >> v2:172.21.15.167:3300/0 conn(0x7f7d340cf7c0 msgr2=0x7f7d340688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 --2- 172.21.15.33:0/3015507756 >> v2:172.21.15.167:3300/0 conn(0x7f7d340cf7c0 0x7f7d340688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 -- 172.21.15.33:0/3015507756 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340ceb40 con 0x7f7d34158c60 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3a322700 1 --2- 172.21.15.33:0/3015507756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34068390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406ff00 tx=0x7f7d30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ea242dc63267d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d39320700 1 -- 172.21.15.33:0/3015507756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30009e60 con 0x7f7d34158c60 2023-10-06T02:46:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d39320700 1 -- 172.21.15.33:0/3015507756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3000b860 con 0x7f7d34158c60 2023-10-06T02:46:13.444 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d39320700 1 -- 172.21.15.33:0/3015507756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000bc20 con 0x7f7d34158c60 2023-10-06T02:46:13.445 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- 172.21.15.33:0/3015507756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 msgr2=0x7f7d34068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- 172.21.15.33:0/3015507756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34068390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406ff00 tx=0x7f7d30009250 comp rx=0 tx=0).stop 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- 172.21.15.33:0/3015507756 shutdown_connections 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- 172.21.15.33:0/3015507756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d3415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- 172.21.15.33:0/3015507756 >> v2:172.21.15.167:3300/0 conn(0x7f7d340cf7c0 0x7f7d340688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 --2- 172.21.15.33:0/3015507756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34068390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- 172.21.15.33:0/3015507756 >> 172.21.15.33:0/3015507756 conn(0x7f7d34067a40 msgr2=0x7f7d340b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- 172.21.15.33:0/3015507756 shutdown_connections 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.437+0000 7f7d3b324700 1 -- 172.21.15.33:0/3015507756 wait complete. 2023-10-06T02:46:13.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 Processor -- start 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- start start 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d340cf7c0 0x7f7d34150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d340b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340b1300 con 0x7f7d34158c60 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34151450 con 0x7f7d340cf7c0 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d341598a0 con 0x7f7d34159030 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53218/0 (socket says 172.21.15.33:53218) 2023-10-06T02:46:13.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 -- 172.21.15.33:0/2716060847 learned_addr learned my addr 172.21.15.33:0/2716060847 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3ab23700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d340b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3a322700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d340cf7c0 0x7f7d34150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 -- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 msgr2=0x7f7d340b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d340b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 -- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d340cf7c0 msgr2=0x7f7d34150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d340cf7c0 0x7f7d34150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 -- 172.21.15.33:0/2716060847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30009490 con 0x7f7d34158c60 2023-10-06T02:46:13.448 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d39b21700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34157680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800ec80 tx=0x7f7d280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800c850 con 0x7f7d34158c60 2023-10-06T02:46:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2800c9b0 con 0x7f7d34158c60 2023-10-06T02:46:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d341584b0 con 0x7f7d34158c60 2023-10-06T02:46:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800cd50 con 0x7f7d34158c60 2023-10-06T02:46:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d34158250 con 0x7f7d34158c60 2023-10-06T02:46:13.449 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.441+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d34159a10 con 0x7f7d34158c60 2023-10-06T02:46:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.445+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7d28013680 con 0x7f7d34158c60 2023-10-06T02:46:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.445+0000 7f7d267fc700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d18060550 0x7f7d18062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.445+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f7d2807d5e0 con 0x7f7d34158c60 2023-10-06T02:46:13.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.445+0000 7f7d3a322700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d18060550 0x7f7d18062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.451 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.445+0000 7f7d3a322700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d18060550 0x7f7d18062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406ff00 tx=0x7f7d30017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.453+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d2804d7c0 con 0x7f7d34158c60 2023-10-06T02:46:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.613+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d34071040 con 0x7f7d34158c60 2023-10-06T02:46:13.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d267fc700 1 -- 172.21.15.33:0/2716060847 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d28050de0 con 0x7f7d34158c60 2023-10-06T02:46:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:13.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d18060550 msgr2=0x7f7d18062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d18060550 0x7f7d18062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d3406ff00 tx=0x7f7d30017000 comp rx=0 tx=0).stop 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 msgr2=0x7f7d34157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34157680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800ec80 tx=0x7f7d280091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 shutdown_connections 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7d18060550 0x7f7d18062a00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d34159030 0x7f7d340b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d34158c60 0x7f7d34157680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 --2- 172.21.15.33:0/2716060847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d340cf7c0 0x7f7d34150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 >> 172.21.15.33:0/2716060847 conn(0x7f7d34067a40 msgr2=0x7f7d3415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 shutdown_connections 2023-10-06T02:46:13.625 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.617+0000 7f7d3b324700 1 -- 172.21.15.33:0/2716060847 wait complete. 2023-10-06T02:46:13.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:46:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.741+0000 7f4d78e8f700 1 Processor -- start 2023-10-06T02:46:13.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.741+0000 7f4d78e8f700 1 -- start start 2023-10-06T02:46:13.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d78e8f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d7415d3b0 0x7f4d740b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.749 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d78e8f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d740cb940 con 0x7f4d74158fc0 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d78e8f700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d740cad30 con 0x7f4d74158bf0 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d78e8f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740b5430 con 0x7f4d7415d3b0 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d71d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4d74158fc0 0x7f4d740b2b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60436/0 (socket says 172.21.15.33:60436) 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d71d9b700 1 -- 172.21.15.33:0/1956844383 learned_addr learned my addr 172.21.15.33:0/1956844383 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d72d9d700 1 --2- 172.21.15.33:0/1956844383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d7415d3b0 0x7f4d740b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648117313 0 0) 0x7f4d740cb940 con 0x7f4d74158fc0 2023-10-06T02:46:13.750 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d60003570 con 0x7f4d74158fc0 2023-10-06T02:46:13.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609761749 0 0) 0x7f4d740cad30 con 0x7f4d74158bf0 2023-10-06T02:46:13.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d740cb940 con 0x7f4d74158bf0 2023-10-06T02:46:13.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d72d9d700 1 -- 172.21.15.33:0/1956844383 >> v1:172.21.15.167:6789/0 conn(0x7f4d74158bf0 legacy=0x7f4d740cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1371687302 0 0) 0x7f4d60003570 con 0x7f4d74158fc0 2023-10-06T02:46:13.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d72d9d700 1 -- 172.21.15.33:0/1956844383 >> v1:172.21.15.33:6789/0 conn(0x7f4d74158fc0 legacy=0x7f4d740b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.752 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d72d9d700 1 -- 172.21.15.33:0/1956844383 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740cb010 con 0x7f4d7415d3b0 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d72d9d700 1 --2- 172.21.15.33:0/1956844383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d7415d3b0 0x7f4d740b1bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009910 tx=0x7f4d6c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781cf34708dd6af9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c001470 con 0x7f4d7415d3b0 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c011420 con 0x7f4d7415d3b0 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d7159a700 1 -- 172.21.15.33:0/1956844383 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0117a0 con 0x7f4d7415d3b0 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.745+0000 7f4d78e8f700 1 -- 172.21.15.33:0/1956844383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d7415d3b0 msgr2=0x7f4d740b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/1956844383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d7415d3b0 0x7f4d740b1bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009910 tx=0x7f4d6c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- 172.21.15.33:0/1956844383 shutdown_connections 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/1956844383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d7415d3b0 0x7f4d740b1bd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- 172.21.15.33:0/1956844383 >> 172.21.15.33:0/1956844383 conn(0x7f4d740cfd70 msgr2=0x7f4d740d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- 172.21.15.33:0/1956844383 shutdown_connections 2023-10-06T02:46:13.754 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- 172.21.15.33:0/1956844383 wait complete. 2023-10-06T02:46:13.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 Processor -- start 2023-10-06T02:46:13.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- start start 2023-10-06T02:46:13.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d74158bf0 0x7f4d741562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d74158fc0 0x7f4d7415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 0x7f4d740ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740c9d60 con 0x7f4d7415d3b0 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740c6f00 con 0x7f4d74158bf0 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d78e8f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740c71e0 con 0x7f4d74158fc0 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 0x7f4d740ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 0x7f4d740ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53230/0 (socket says 172.21.15.33:53230) 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 -- 172.21.15.33:0/361155584 learned_addr learned my addr 172.21.15.33:0/361155584 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d7259c700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d74158bf0 0x7f4d741562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d71d9b700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d74158fc0 0x7f4d7415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 -- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d74158fc0 msgr2=0x7f4d7415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d74158fc0 0x7f4d7415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 -- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d74158bf0 msgr2=0x7f4d741562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d74158bf0 0x7f4d741562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d72d9d700 1 -- 172.21.15.33:0/361155584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009420 con 0x7f4d7415d3b0 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.749+0000 7f4d71d9b700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d74158fc0 0x7f4d7415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d72d9d700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 0x7f4d740ce7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009f00 tx=0x7f4d6c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c01cdf0 con 0x7f4d7415d3b0 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c01a450 con 0x7f4d7415d3b0 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d740c74c0 con 0x7f4d7415d3b0 2023-10-06T02:46:13.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c01a810 con 0x7f4d7415d3b0 2023-10-06T02:46:13.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7415ae20 con 0x7f4d7415d3b0 2023-10-06T02:46:13.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740c76d0 con 0x7f4d7415d3b0 2023-10-06T02:46:13.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4d6c01ac00 con 0x7f4d7415d3b0 2023-10-06T02:46:13.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.753+0000 7f4d6a7fc700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4d54060550 0x7f4d54062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:13.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.757+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f4d6c07e670 con 0x7f4d7415d3b0 2023-10-06T02:46:13.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.757+0000 7f4d7259c700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4d54060550 0x7f4d54062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:13.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.757+0000 7f4d7259c700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4d54060550 0x7f4d54062a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d74169bf0 tx=0x7f4d64007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:13.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.761+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4d6c04e730 con 0x7f4d7415d3b0 2023-10-06T02:46:13.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.925+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d7415b070 con 0x7f4d7415d3b0 2023-10-06T02:46:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.925+0000 7f4d6a7fc700 1 -- 172.21.15.33:0/361155584 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d6c051d50 con 0x7f4d7415d3b0 2023-10-06T02:46:13.931 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:46:13.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4d54060550 msgr2=0x7f4d54062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4d54060550 0x7f4d54062a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d74169bf0 tx=0x7f4d64007f90 comp rx=0 tx=0).stop 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 msgr2=0x7f4d740ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 0x7f4d740ce7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009f00 tx=0x7f4d6c016000 comp rx=0 tx=0).stop 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 shutdown_connections 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4d54060550 0x7f4d54062a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d74158fc0 0x7f4d7415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d7415d3b0 0x7f4d740ce7e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 --2- 172.21.15.33:0/361155584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d74158bf0 0x7f4d741562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 >> 172.21.15.33:0/361155584 conn(0x7f4d740cfd70 msgr2=0x7f4d741581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 shutdown_connections 2023-10-06T02:46:13.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:13.929+0000 7f4d78e8f700 1 -- 172.21.15.33:0/361155584 wait complete. 2023-10-06T02:46:13.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-06T02:46:14.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f82663b0700 1 Processor -- start 2023-10-06T02:46:14.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f82663b0700 1 -- start start 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f82663b0700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8260159030 0x7f82600caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f82663b0700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600cd7f0 con 0x7f82600cb020 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f82663b0700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600cddb0 con 0x7f8260158c60 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f82663b0700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8260159480 con 0x7f8260159030 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8265baf700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f82600cb020 0x7f82600ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60448/0 (socket says 172.21.15.33:60448) 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8265baf700 1 -- 172.21.15.33:0/1556124792 learned_addr learned my addr 172.21.15.33:0/1556124792 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8264bad700 1 --2- 172.21.15.33:0/1556124792 >> v2:172.21.15.167:3300/0 conn(0x7f8260159030 0x7f82600caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8257fff700 1 -- 172.21.15.33:0/1556124792 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214096903 0 0) 0x7f82600cd7f0 con 0x7f82600cb020 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8257fff700 1 -- 172.21.15.33:0/1556124792 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f824c003570 con 0x7f82600cb020 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8264bad700 1 -- 172.21.15.33:0/1556124792 >> v1:172.21.15.167:6789/0 conn(0x7f8260158c60 legacy=0x7f826015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8264bad700 1 -- 172.21.15.33:0/1556124792 >> v1:172.21.15.33:6789/0 conn(0x7f82600cb020 legacy=0x7f82600ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8257fff700 1 -- 172.21.15.33:0/1556124792 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132108680 0 0) 0x7f824c003570 con 0x7f82600cb020 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8264bad700 1 -- 172.21.15.33:0/1556124792 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826015d630 con 0x7f8260159030 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8264bad700 1 --2- 172.21.15.33:0/1556124792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f82600caae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82500098e0 tx=0x7f82500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb0246b718dbd918 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8257fff700 1 -- 172.21.15.33:0/1556124792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000b640 con 0x7f8260159030 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8257fff700 1 -- 172.21.15.33:0/1556124792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000b7a0 con 0x7f8260159030 2023-10-06T02:46:14.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.057+0000 7f8257fff700 1 -- 172.21.15.33:0/1556124792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000bb40 con 0x7f8260159030 2023-10-06T02:46:14.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 -- 172.21.15.33:0/1556124792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 msgr2=0x7f82600caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 --2- 172.21.15.33:0/1556124792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f82600caae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82500098e0 tx=0x7f82500091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 -- 172.21.15.33:0/1556124792 shutdown_connections 2023-10-06T02:46:14.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 --2- 172.21.15.33:0/1556124792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f82600caae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 -- 172.21.15.33:0/1556124792 >> 172.21.15.33:0/1556124792 conn(0x7f8260067a80 msgr2=0x7f82600cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:14.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 -- 172.21.15.33:0/1556124792 shutdown_connections 2023-10-06T02:46:14.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.061+0000 7f82663b0700 1 -- 172.21.15.33:0/1556124792 wait complete. 2023-10-06T02:46:14.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 Processor -- start 2023-10-06T02:46:14.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 -- start start 2023-10-06T02:46:14.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82600cb020 0x7f82600c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 0x7f82600b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f8260149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c9760 con 0x7f8260158c60 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c86c0 con 0x7f8260159030 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c89a0 con 0x7f82600cb020 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 0x7f82600b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 0x7f82600b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53232/0 (socket says 172.21.15.33:53232) 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 -- 172.21.15.33:0/2794321050 learned_addr learned my addr 172.21.15.33:0/2794321050 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8265baf700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f8260149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82653ae700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82600cb020 0x7f82600c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 -- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82600cb020 msgr2=0x7f82600c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82600cb020 0x7f82600c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 -- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 msgr2=0x7f8260149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f8260149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 -- 172.21.15.33:0/2794321050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009420 con 0x7f8260158c60 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f8264bad700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 0x7f82600b1470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82500093f0 tx=0x7f8250017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825001f070 con 0x7f8260158c60 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250000bc0 con 0x7f8260158c60 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.065+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82600c8c80 con 0x7f8260158c60 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250025420 con 0x7f8260158c60 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82600c9240 con 0x7f8260158c60 2023-10-06T02:46:14.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8260071040 con 0x7f8260158c60 2023-10-06T02:46:14.074 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8250023070 con 0x7f8260158c60 2023-10-06T02:46:14.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82557fa700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8244060550 0x7f8244062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.075 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f825007df70 con 0x7f8260158c60 2023-10-06T02:46:14.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.069+0000 7f82653ae700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8244060550 0x7f8244062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.073+0000 7f82653ae700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8244060550 0x7f8244062a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f825c003a80 tx=0x7f825c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.077+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f82500adbe0 con 0x7f8260158c60 2023-10-06T02:46:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f826015d910 con 0x7f8260158c60 2023-10-06T02:46:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82557fa700 1 -- 172.21.15.33:0/2794321050 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v32) v1 ==== 103+0+11 (secure 0 0 0) 0x7f825002a020 con 0x7f8260158c60 2023-10-06T02:46:14.245 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8244060550 msgr2=0x7f8244062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8244060550 0x7f8244062a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f825c003a80 tx=0x7f825c007ec0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 msgr2=0x7f82600b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 0x7f82600b1470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82500093f0 tx=0x7f8250017000 comp rx=0 tx=0).stop 2023-10-06T02:46:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 shutdown_connections 2023-10-06T02:46:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8260159030 0x7f8260149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8260158c60 0x7f82600b1470 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8244060550 0x7f8244062a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 --2- 172.21.15.33:0/2794321050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82600cb020 0x7f82600c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 >> 172.21.15.33:0/2794321050 conn(0x7f8260067a80 msgr2=0x7f82601547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:14.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 shutdown_connections 2023-10-06T02:46:14.249 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.241+0000 7f82663b0700 1 -- 172.21.15.33:0/2794321050 wait complete. 2023-10-06T02:46:14.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:14.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 Processor -- start 2023-10-06T02:46:14.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- start start 2023-10-06T02:46:14.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc340680d0 0x7fdc340aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc34157650 0x7fdc34157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc340cb6e0 con 0x7fdc34157280 2023-10-06T02:46:14.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc340c6c10 con 0x7fdc34157650 2023-10-06T02:46:14.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34158010 con 0x7fdc340680d0 2023-10-06T02:46:14.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc32d9d700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdc34157650 0x7fdc34157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3259c700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fdc34157280 0x7fdc340679c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:58338/0 (socket says 172.21.15.33:58338) 2023-10-06T02:46:14.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3259c700 1 -- 172.21.15.33:0/40058076 learned_addr learned my addr 172.21.15.33:0/40058076 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:14.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc29d9b700 1 --2- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc340680d0 0x7fdc340aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc31d9b700 1 -- 172.21.15.33:0/40058076 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395681935 0 0) 0x7fdc340cb6e0 con 0x7fdc34157280 2023-10-06T02:46:14.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc31d9b700 1 -- 172.21.15.33:0/40058076 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc20003570 con 0x7fdc34157280 2023-10-06T02:46:14.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc32d9d700 1 -- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc340680d0 msgr2=0x7fdc340aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc32d9d700 1 --2- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc340680d0 0x7fdc340aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc32d9d700 1 -- 172.21.15.33:0/40058076 >> v1:172.21.15.167:6789/0 conn(0x7fdc34157280 legacy=0x7fdc340679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc32d9d700 1 -- 172.21.15.33:0/40058076 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc340685d0 con 0x7fdc34157650 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc32d9d700 1 --2- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc34157650 0x7fdc34157ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009e60 tx=0x7fdc2c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d300870f434761e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc31d9b700 1 -- 172.21.15.33:0/40058076 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c00e070 con 0x7fdc34157650 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc31d9b700 1 -- 172.21.15.33:0/40058076 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c00bab0 con 0x7fdc34157650 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc31d9b700 1 -- 172.21.15.33:0/40058076 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c00be70 con 0x7fdc34157650 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc34157650 msgr2=0x7fdc34157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 --2- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc34157650 0x7fdc34157ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009e60 tx=0x7fdc2c009780 comp rx=0 tx=0).stop 2023-10-06T02:46:14.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- 172.21.15.33:0/40058076 shutdown_connections 2023-10-06T02:46:14.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 --2- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc34157650 0x7fdc34157ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 --2- 172.21.15.33:0/40058076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc340680d0 0x7fdc340aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- 172.21.15.33:0/40058076 >> 172.21.15.33:0/40058076 conn(0x7fdc340c83f0 msgr2=0x7fdc340c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:14.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- 172.21.15.33:0/40058076 shutdown_connections 2023-10-06T02:46:14.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.365+0000 7fdc3886c700 1 -- 172.21.15.33:0/40058076 wait complete. 2023-10-06T02:46:14.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 Processor -- start 2023-10-06T02:46:14.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 -- start start 2023-10-06T02:46:14.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc340680d0 0x7fdc3414cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 0x7fdc340c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc34157650 0x7fdc34156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340af660 con 0x7fdc34157280 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340af940 con 0x7fdc340680d0 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340afc20 con 0x7fdc34157650 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 0x7fdc340c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 0x7fdc340c54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53246/0 (socket says 172.21.15.33:53246) 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 -- 172.21.15.33:0/3435166259 learned_addr learned my addr 172.21.15.33:0/3435166259 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc32d9d700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc34157650 0x7fdc34156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3259c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc340680d0 0x7fdc3414cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 -- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc34157650 msgr2=0x7fdc34156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc34157650 0x7fdc34156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 -- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc340680d0 msgr2=0x7fdc3414cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc340680d0 0x7fdc3414cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 -- 172.21.15.33:0/3435166259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c009a20 con 0x7fdc34157280 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc32d9d700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc34157650 0x7fdc34156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc29d9b700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 0x7fdc340c54f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00ec80 tx=0x7fdc1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00c850 con 0x7fdc34157280 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c00c9b0 con 0x7fdc34157280 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc340aff00 con 0x7fdc34157280 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.369+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00cd50 con 0x7fdc34157280 2023-10-06T02:46:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.373+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3414e810 con 0x7fdc34157280 2023-10-06T02:46:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.373+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3414eac0 con 0x7fdc34157280 2023-10-06T02:46:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.373+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdc1c013680 con 0x7fdc34157280 2023-10-06T02:46:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.373+0000 7fdc2affd700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc14060580 0x7fdc14062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.373+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fdc1c07ce80 con 0x7fdc34157280 2023-10-06T02:46:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.373+0000 7fdc3259c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc14060580 0x7fdc14062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.377+0000 7fdc3259c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc14060580 0x7fdc14062a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc24007fd0 tx=0x7fdc24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.385 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.381+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdc3414eac0 con 0x7fdc34157280 2023-10-06T02:46:14.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.581+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc3414eac0 con 0x7fdc34157280 2023-10-06T02:46:14.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.581+0000 7fdc2affd700 1 -- 172.21.15.33:0/3435166259 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fdc3414eac0 con 0x7fdc34157280 2023-10-06T02:46:14.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:14.586 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:46:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc14060580 msgr2=0x7fdc14062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc14060580 0x7fdc14062a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc24007fd0 tx=0x7fdc24009040 comp rx=0 tx=0).stop 2023-10-06T02:46:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 msgr2=0x7fdc340c54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 0x7fdc340c54f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00ec80 tx=0x7fdc1c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 shutdown_connections 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdc14060580 0x7fdc14062a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc34157650 0x7fdc34156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc34157280 0x7fdc340c54f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 --2- 172.21.15.33:0/3435166259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc340680d0 0x7fdc3414cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 >> 172.21.15.33:0/3435166259 conn(0x7fdc340c83f0 msgr2=0x7fdc340ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 shutdown_connections 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.585+0000 7fdc3886c700 1 -- 172.21.15.33:0/3435166259 wait complete. 2023-10-06T02:46:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:46:14.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 Processor -- start 2023-10-06T02:46:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 -- start start 2023-10-06T02:46:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 0x7fb4280c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4280c5940 0x7fb4280c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42806add0 con 0x7fb428066fb0 2023-10-06T02:46:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4280c80c0 con 0x7fb4280c5940 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42dd16700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280c7c40 con 0x7fb4280ade00 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb427fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4280c5940 0x7fb4280c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42cd14700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 0x7fb4280c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb427fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb4280c5940 0x7fb4280c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:51760/0 (socket says 172.21.15.33:51760) 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42cd14700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 0x7fb4280c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:56766/0 (socket says 172.21.15.33:56766) 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42d515700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb428066fb0 0x7fb4280c9550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:58346/0 (socket says 172.21.15.33:58346) 2023-10-06T02:46:14.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.701+0000 7fb42cd14700 1 -- 172.21.15.33:0/133221634 learned_addr learned my addr 172.21.15.33:0/133221634 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:14.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb427fff700 1 -- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 msgr2=0x7fb4280c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:14.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb427fff700 1 --2- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 0x7fb4280c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb427fff700 1 -- 172.21.15.33:0/133221634 >> v1:172.21.15.167:6789/0 conn(0x7fb428066fb0 legacy=0x7fb4280c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb427fff700 1 -- 172.21.15.33:0/133221634 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280ae1d0 con 0x7fb4280c5940 2023-10-06T02:46:14.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42cd14700 1 --2- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 0x7fb4280c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:14.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb427fff700 1 --2- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280c5940 0x7fb4280c7700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb4180098e0 tx=0x7fb4180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a6c6d9e1f7b70a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb4277fe700 1 -- 172.21.15.33:0/133221634 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800b640 con 0x7fb4280c5940 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb4277fe700 1 -- 172.21.15.33:0/133221634 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41800b7a0 con 0x7fb4280c5940 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb4277fe700 1 -- 172.21.15.33:0/133221634 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800bb40 con 0x7fb4280c5940 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280c5940 msgr2=0x7fb4280c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 --2- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280c5940 0x7fb4280c7700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb4180098e0 tx=0x7fb4180091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- 172.21.15.33:0/133221634 shutdown_connections 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 --2- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280c5940 0x7fb4280c7700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 --2- 172.21.15.33:0/133221634 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4280ade00 0x7fb4280c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- 172.21.15.33:0/133221634 >> 172.21.15.33:0/133221634 conn(0x7fb4280c8240 msgr2=0x7fb4280c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- 172.21.15.33:0/133221634 shutdown_connections 2023-10-06T02:46:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- 172.21.15.33:0/133221634 wait complete. 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 Processor -- start 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- start start 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb428066fb0 0x7fb4280c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280ade00 0x7fb4280c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 0x7fb4280af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280c4d60 con 0x7fb4280c5940 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280c3ef0 con 0x7fb4280ade00 2023-10-06T02:46:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.705+0000 7fb42dd16700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280c4150 con 0x7fb428066fb0 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 0x7fb4280af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 0x7fb4280af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53258/0 (socket says 172.21.15.33:53258) 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 -- 172.21.15.33:0/1844794993 learned_addr learned my addr 172.21.15.33:0/1844794993 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb427fff700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280ade00 0x7fb4280c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42cd14700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb428066fb0 0x7fb4280c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 -- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb428066fb0 msgr2=0x7fb4280c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb428066fb0 0x7fb4280c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 -- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280ade00 msgr2=0x7fb4280c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280ade00 0x7fb4280c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 -- 172.21.15.33:0/1844794993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0094f0 con 0x7fb4280c5940 2023-10-06T02:46:14.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42cd14700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb428066fb0 0x7fb4280c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42d515700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 0x7fb4280af4c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb42000cc70 tx=0x7fb420001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420005df0 con 0x7fb4280c5940 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42000a660 con 0x7fb4280c5940 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42000a9e0 con 0x7fb4280c5940 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb418009420 con 0x7fb4280c5940 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4280c4710 con 0x7fb4280c5940 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4280c4ed0 con 0x7fb4280c5940 2023-10-06T02:46:14.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.709+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb4200070f0 con 0x7fb4280c5940 2023-10-06T02:46:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.713+0000 7fb424ff9700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb414060580 0x7fb414062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.713+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fb42007cca0 con 0x7fb4280c5940 2023-10-06T02:46:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.713+0000 7fb42cd14700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb414060580 0x7fb414062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:14.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.713+0000 7fb42cd14700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb414060580 0x7fb414062a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb41c006d40 tx=0x7fb41c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.717+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb420050910 con 0x7fb4280c5940 2023-10-06T02:46:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.885+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb428071040 con 0x7fb4280c5940 2023-10-06T02:46:14.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb424ff9700 1 -- 172.21.15.33:0/1844794993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18803 (secure 0 0 0) 0x7fb4200504a0 con 0x7fb4280c5940 2023-10-06T02:46:14.892 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:14.892 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:05.355164+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb414060580 msgr2=0x7fb414062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb414060580 0x7fb414062a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb41c006d40 tx=0x7fb41c008040 comp rx=0 tx=0).stop 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 msgr2=0x7fb4280af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 0x7fb4280af4c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb42000cc70 tx=0x7fb420001a70 comp rx=0 tx=0).stop 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 shutdown_connections 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4280c5940 0x7fb4280af4c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4280ade00 0x7fb4280c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb414060580 0x7fb414062a30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 --2- 172.21.15.33:0/1844794993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb428066fb0 0x7fb4280c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 >> 172.21.15.33:0/1844794993 conn(0x7fb4280c8240 msgr2=0x7fb4280c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 shutdown_connections 2023-10-06T02:46:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:14.889+0000 7fb42dd16700 1 -- 172.21.15.33:0/1844794993 wait complete. 2023-10-06T02:46:14.908 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:46:14.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 Processor -- start 2023-10-06T02:46:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 -- start start 2023-10-06T02:46:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faf70159000 0x7faf700c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf701593d0 0x7faf7006b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf7006a380 0x7faf7006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf700c9c30 con 0x7faf7006a380 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf700c9020 con 0x7faf70159000 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf758b1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf700c9300 con 0x7faf701593d0 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf6f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf7006a380 0x7faf7006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.005+0000 7faf6f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf7006a380 0x7faf7006bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53270/0 (socket says 172.21.15.33:53270) 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 -- 172.21.15.33:0/2734775826 learned_addr learned my addr 172.21.15.33:0/2734775826 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6effd700 1 --2- 172.21.15.33:0/2734775826 >> v2:172.21.15.167:3300/0 conn(0x7faf70159000 0x7faf700c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.012 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf67fff700 1 --2- 172.21.15.33:0/2734775826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf701593d0 0x7faf7006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 -- 172.21.15.33:0/2734775826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf701593d0 msgr2=0x7faf7006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 --2- 172.21.15.33:0/2734775826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf701593d0 0x7faf7006b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 -- 172.21.15.33:0/2734775826 >> v2:172.21.15.167:3300/0 conn(0x7faf70159000 msgr2=0x7faf700c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 --2- 172.21.15.33:0/2734775826 >> v2:172.21.15.167:3300/0 conn(0x7faf70159000 0x7faf700c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 -- 172.21.15.33:0/2734775826 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700c95e0 con 0x7faf7006a380 2023-10-06T02:46:15.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6effd700 1 --2- 172.21.15.33:0/2734775826 >> v2:172.21.15.167:3300/0 conn(0x7faf70159000 0x7faf700c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6f7fe700 1 --2- 172.21.15.33:0/2734775826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf7006bca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faf70087050 tx=0x7faf68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f110e28b531d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6e7fc700 1 -- 172.21.15.33:0/2734775826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6800e070 con 0x7faf7006a380 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6e7fc700 1 -- 172.21.15.33:0/2734775826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6800b970 con 0x7faf7006a380 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf6e7fc700 1 -- 172.21.15.33:0/2734775826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680092e0 con 0x7faf7006a380 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 -- 172.21.15.33:0/2734775826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 msgr2=0x7faf7006bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 --2- 172.21.15.33:0/2734775826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf7006bca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faf70087050 tx=0x7faf68009840 comp rx=0 tx=0).stop 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 -- 172.21.15.33:0/2734775826 shutdown_connections 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 --2- 172.21.15.33:0/2734775826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf7006bca0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 --2- 172.21.15.33:0/2734775826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf701593d0 0x7faf7006b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 --2- 172.21.15.33:0/2734775826 >> v2:172.21.15.167:3300/0 conn(0x7faf70159000 0x7faf700c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 -- 172.21.15.33:0/2734775826 >> 172.21.15.33:0/2734775826 conn(0x7faf700af920 msgr2=0x7faf700afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 -- 172.21.15.33:0/2734775826 shutdown_connections 2023-10-06T02:46:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.009+0000 7faf758b1700 1 -- 172.21.15.33:0/2734775826 wait complete. 2023-10-06T02:46:15.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 Processor -- start 2023-10-06T02:46:15.017 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 -- start start 2023-10-06T02:46:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf70156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf70159000 0x7faf70099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf701593d0 0x7faf70067de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf701565a0 con 0x7faf7006a380 2023-10-06T02:46:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7009a480 con 0x7faf701593d0 2023-10-06T02:46:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf758b1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf700c04b0 con 0x7faf70159000 2023-10-06T02:46:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf70156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf70156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53282/0 (socket says 172.21.15.33:53282) 2023-10-06T02:46:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 -- 172.21.15.33:0/2391216265 learned_addr learned my addr 172.21.15.33:0/2391216265 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6f7fe700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf701593d0 0x7faf70067de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf67fff700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf70159000 0x7faf70099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 -- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf70159000 msgr2=0x7faf70099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf70159000 0x7faf70099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 -- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf701593d0 msgr2=0x7faf70067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf701593d0 0x7faf70067de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 -- 172.21.15.33:0/2391216265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600093c0 con 0x7faf7006a380 2023-10-06T02:46:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf6effd700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf70156060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf6000ec80 tx=0x7faf60009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.013+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60005a00 con 0x7faf7006a380 2023-10-06T02:46:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.017+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf60005b60 con 0x7faf7006a380 2023-10-06T02:46:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.017+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf68009ae0 con 0x7faf7006a380 2023-10-06T02:46:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.017+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000c9d0 con 0x7faf7006a380 2023-10-06T02:46:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.017+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf700c0a70 con 0x7faf7006a380 2023-10-06T02:46:15.025 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.017+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf70063f20 con 0x7faf7006a380 2023-10-06T02:46:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.017+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faf6000cd80 con 0x7faf7006a380 2023-10-06T02:46:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.021+0000 7faf66ffd700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faf50060550 0x7faf50062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.021+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7faf6007d230 con 0x7faf7006a380 2023-10-06T02:46:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.021+0000 7faf67fff700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faf50060550 0x7faf50062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.021+0000 7faf67fff700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faf50060550 0x7faf50062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf700cbeb0 tx=0x7faf58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.025+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faf6004d390 con 0x7faf7006a380 2023-10-06T02:46:15.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.221+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf700710b0 con 0x7faf7006a380 2023-10-06T02:46:15.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf66ffd700 1 -- 172.21.15.33:0/2391216265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7faf600509b0 con 0x7faf7006a380 2023-10-06T02:46:15.229 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:15.229 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:46:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faf50060550 msgr2=0x7faf50062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faf50060550 0x7faf50062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf700cbeb0 tx=0x7faf58008040 comp rx=0 tx=0).stop 2023-10-06T02:46:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 msgr2=0x7faf70156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf70156060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf6000ec80 tx=0x7faf60009120 comp rx=0 tx=0).stop 2023-10-06T02:46:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 shutdown_connections 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faf50060550 0x7faf50062a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf70159000 0x7faf70099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf701593d0 0x7faf70067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 --2- 172.21.15.33:0/2391216265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf7006a380 0x7faf70156060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 >> 172.21.15.33:0/2391216265 conn(0x7faf700af920 msgr2=0x7faf7006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 shutdown_connections 2023-10-06T02:46:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.225+0000 7faf758b1700 1 -- 172.21.15.33:0/2391216265 wait complete. 2023-10-06T02:46:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:46:15.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:15.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19fa205700 1 Processor -- start 2023-10-06T02:46:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19fa205700 1 -- start start 2023-10-06T02:46:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19fa205700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f40cf420 0x7f19f415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19fa205700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40cdf80 con 0x7f19f4068b60 2023-10-06T02:46:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19fa205700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f415d730 con 0x7f19f4158d00 2023-10-06T02:46:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19fa205700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f415da10 con 0x7f19f40cf420 2023-10-06T02:46:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19f9a04700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f19f4068b60 0x7f19f41590d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60456/0 (socket says 172.21.15.33:60456) 2023-10-06T02:46:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19f9a04700 1 -- 172.21.15.33:0/939287385 learned_addr learned my addr 172.21.15.33:0/939287385 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.349+0000 7f19f8a02700 1 --2- 172.21.15.33:0/939287385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f40cf420 0x7f19f415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033765894 0 0) 0x7f19f415d730 con 0x7f19f4158d00 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19dc003570 con 0x7f19f4158d00 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429376480 0 0) 0x7f19f40cdf80 con 0x7f19f4068b60 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f415d730 con 0x7f19f4068b60 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f8a02700 1 -- 172.21.15.33:0/939287385 >> v1:172.21.15.167:6789/0 conn(0x7f19f4158d00 legacy=0x7f19f4068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f8a02700 1 -- 172.21.15.33:0/939287385 >> v1:172.21.15.33:6789/0 conn(0x7f19f4068b60 legacy=0x7f19f41590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f8a02700 1 -- 172.21.15.33:0/939287385 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40cd930 con 0x7f19f40cf420 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f8a02700 1 --2- 172.21.15.33:0/939287385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f40cf420 0x7f19f415ca70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009910 tx=0x7f19e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c8ca20d77b619d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8011630 con 0x7f19f40cf420 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e8011790 con 0x7f19f40cf420 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19f3fff700 1 -- 172.21.15.33:0/939287385 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8011b30 con 0x7f19f40cf420 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 -- 172.21.15.33:0/939287385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f40cf420 msgr2=0x7f19f415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 --2- 172.21.15.33:0/939287385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f40cf420 0x7f19f415ca70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009910 tx=0x7f19e80091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 -- 172.21.15.33:0/939287385 shutdown_connections 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 --2- 172.21.15.33:0/939287385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f40cf420 0x7f19f415ca70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 -- 172.21.15.33:0/939287385 >> 172.21.15.33:0/939287385 conn(0x7f19f4067a80 msgr2=0x7f19f40cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 -- 172.21.15.33:0/939287385 shutdown_connections 2023-10-06T02:46:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 -- 172.21.15.33:0/939287385 wait complete. 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 Processor -- start 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.353+0000 7f19fa205700 1 -- start start 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 0x7f19f40cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f40cf420 0x7f19f414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f4158d00 0x7f19f415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f414fb80 con 0x7f19f4068b60 2023-10-06T02:46:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41539f0 con 0x7f19f40cf420 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cd330 con 0x7f19f4158d00 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 0x7f19f40cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 0x7f19f40cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53294/0 (socket says 172.21.15.33:53294) 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 -- 172.21.15.33:0/4101348576 learned_addr learned my addr 172.21.15.33:0/4101348576 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f8a02700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f40cf420 0x7f19f414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9a04700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f4158d00 0x7f19f415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 -- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f4158d00 msgr2=0x7f19f415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f4158d00 0x7f19f415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 -- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f40cf420 msgr2=0x7f19f414f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f40cf420 0x7f19f414f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 -- 172.21.15.33:0/4101348576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e8009420 con 0x7f19f4068b60 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f9203700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 0x7f19f40cccc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19e400c580 tx=0x7f19e4007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4010510 con 0x7f19f4068b60 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4010670 con 0x7f19f4068b60 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f4152c30 con 0x7f19f4068b60 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4010a30 con 0x7f19f4068b60 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f41531f0 con 0x7f19f4068b60 2023-10-06T02:46:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f4071040 con 0x7f19f4068b60 2023-10-06T02:46:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f19e401b400 con 0x7f19f4068b60 2023-10-06T02:46:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f17fa700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19e0060550 0x7f19e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f19e407c740 con 0x7f19f4068b60 2023-10-06T02:46:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.357+0000 7f19f8a02700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19e0060550 0x7f19e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.361+0000 7f19f8a02700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19e0060550 0x7f19e0062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009860 tx=0x7f19e8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.365+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19e404c8a0 con 0x7f19f4068b60 2023-10-06T02:46:15.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.561+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f40cdc10 con 0x7f19f4068b60 2023-10-06T02:46:15.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.561+0000 7f19f17fa700 1 -- 172.21.15.33:0/4101348576 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f19e404fec0 con 0x7f19f4068b60 2023-10-06T02:46:15.567 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:15.567 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2308745571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2308745571},{"type":"v1","addr":"172.21.15.167:6835","nonce":2308745571}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1939556557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":1939556557},{"type":"v1","addr":"172.21.15.167:6837","nonce":1939556557}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/3556551963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3556551963},{"type":"v1","addr":"172.21.15.33:6833","nonce":3556551963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:45:23.146455+0000","modified":"2023-10-06T02:45:41.377151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/2898450531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2898450531},{"type":"v1","addr":"172.21.15.33:6835","nonce":2898450531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19e0060550 msgr2=0x7f19e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19e0060550 0x7f19e0062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009860 tx=0x7f19e8017000 comp rx=0 tx=0).stop 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 msgr2=0x7f19f40cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 0x7f19f40cccc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19e400c580 tx=0x7f19e4007e30 comp rx=0 tx=0).stop 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 shutdown_connections 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19e0060550 0x7f19e0062a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19f4158d00 0x7f19f415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19f40cf420 0x7f19f414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 --2- 172.21.15.33:0/4101348576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19f4068b60 0x7f19f40cccc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 >> 172.21.15.33:0/4101348576 conn(0x7f19f4067a80 msgr2=0x7f19f415f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:15.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 shutdown_connections 2023-10-06T02:46:15.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.565+0000 7f19fa205700 1 -- 172.21.15.33:0/4101348576 wait complete. 2023-10-06T02:46:15.572 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2023-10-06T02:46:15.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.689+0000 7f11f910b700 1 Processor -- start 2023-10-06T02:46:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.689+0000 7f11f910b700 1 -- start start 2023-10-06T02:46:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f11f415d680 0x7f11f4153060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11f41535a0 0x7f11f4159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f40b3860 con 0x7f11f40af560 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f415db20 con 0x7f11f41535a0 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f415d1f0 con 0x7f11f415d680 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11f41535a0 0x7f11f4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f11f41535a0 0x7f11f4159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53300/0 (socket says 172.21.15.33:53300) 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 -- 172.21.15.33:0/3419407786 learned_addr learned my addr 172.21.15.33:0/3419407786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f259c700 1 --2- 172.21.15.33:0/3419407786 >> v2:172.21.15.167:3300/0 conn(0x7f11f415d680 0x7f11f4153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 -- 172.21.15.33:0/3419407786 >> v2:172.21.15.167:3300/0 conn(0x7f11f415d680 msgr2=0x7f11f4153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 --2- 172.21.15.33:0/3419407786 >> v2:172.21.15.167:3300/0 conn(0x7f11f415d680 0x7f11f4153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 -- 172.21.15.33:0/3419407786 >> v1:172.21.15.167:6789/0 conn(0x7f11f40af560 legacy=0x7f11f40af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 -- 172.21.15.33:0/3419407786 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f415cbe0 con 0x7f11f41535a0 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f359e700 1 --2- 172.21.15.33:0/3419407786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f41535a0 0x7f11f4159540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11f40684e0 tx=0x7f11ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b368cd227d477db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f1d9b700 1 -- 172.21.15.33:0/3419407786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00e070 con 0x7f11f41535a0 2023-10-06T02:46:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f1d9b700 1 -- 172.21.15.33:0/3419407786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec00ba20 con 0x7f11f41535a0 2023-10-06T02:46:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f1d9b700 1 -- 172.21.15.33:0/3419407786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0092e0 con 0x7f11f41535a0 2023-10-06T02:46:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- 172.21.15.33:0/3419407786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f41535a0 msgr2=0x7f11f4159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 --2- 172.21.15.33:0/3419407786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f41535a0 0x7f11f4159540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11f40684e0 tx=0x7f11ec009840 comp rx=0 tx=0).stop 2023-10-06T02:46:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- 172.21.15.33:0/3419407786 shutdown_connections 2023-10-06T02:46:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 --2- 172.21.15.33:0/3419407786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f41535a0 0x7f11f4159540 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 --2- 172.21.15.33:0/3419407786 >> v2:172.21.15.167:3300/0 conn(0x7f11f415d680 0x7f11f4153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- 172.21.15.33:0/3419407786 >> 172.21.15.33:0/3419407786 conn(0x7f11f40cdb90 msgr2=0x7f11f40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- 172.21.15.33:0/3419407786 shutdown_connections 2023-10-06T02:46:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.693+0000 7f11f910b700 1 -- 172.21.15.33:0/3419407786 wait complete. 2023-10-06T02:46:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.697+0000 7f11f910b700 1 Processor -- start 2023-10-06T02:46:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.697+0000 7f11f910b700 1 -- start start 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.697+0000 7f11f910b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 0x7f11f41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f41535a0 0x7f11f40c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f415d680 0x7f11f416ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4156290 con 0x7f11f40af560 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4155680 con 0x7f11f41535a0 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4155960 con 0x7f11f415d680 2023-10-06T02:46:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 0x7f11f41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 0x7f11f41511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53310/0 (socket says 172.21.15.33:53310) 2023-10-06T02:46:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 -- 172.21.15.33:0/1266841979 learned_addr learned my addr 172.21.15.33:0/1266841979 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f259c700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f41535a0 0x7f11f40c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f359e700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f415d680 0x7f11f416ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 -- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f415d680 msgr2=0x7f11f416ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f415d680 0x7f11f416ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 -- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f41535a0 msgr2=0x7f11f40c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f41535a0 0x7f11f40c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 -- 172.21.15.33:0/1266841979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec009ae0 con 0x7f11f40af560 2023-10-06T02:46:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f2d9d700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 0x7f11f41511e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c590 tx=0x7f11e400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4011590 con 0x7f11f40af560 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4011710 con 0x7f11f40af560 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f4155c40 con 0x7f11f40af560 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4011ad0 con 0x7f11f40af560 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f4155300 con 0x7f11f40af560 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f40710b0 con 0x7f11f40af560 2023-10-06T02:46:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.701+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f11e401b3b0 con 0x7f11f40af560 2023-10-06T02:46:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.705+0000 7f11eaffd700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f11d4060550 0x7f11d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.705+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f11e407c0c0 con 0x7f11f40af560 2023-10-06T02:46:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.705+0000 7f11f259c700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f11d4060550 0x7f11d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.705+0000 7f11f259c700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f11d4060550 0x7f11d4062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ae1b0 tx=0x7f11dc0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.713+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f11e404c220 con 0x7f11f40af560 2023-10-06T02:46:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.877+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f415a190 con 0x7f11f40af560 2023-10-06T02:46:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11eaffd700 1 -- 172.21.15.33:0/1266841979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+18803 (secure 0 0 0) 0x7f11e4017090 con 0x7f11f40af560 2023-10-06T02:46:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:15.884 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:05.355164+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:45:21.485516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T02:45:22.142461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-10-06T02:45:25.344940+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.19","mappings":[{"from":3,"to":5}]},{"pgid":"4.39","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f11d4060550 msgr2=0x7f11d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f11d4060550 0x7f11d4062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ae1b0 tx=0x7f11dc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 msgr2=0x7f11f41511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 0x7f11f41511e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c590 tx=0x7f11e400c8a0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 shutdown_connections 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f11d4060550 0x7f11d4062a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11f415d680 0x7f11f416ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11f41535a0 0x7f11f40c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 --2- 172.21.15.33:0/1266841979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11f40af560 0x7f11f41511e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 >> 172.21.15.33:0/1266841979 conn(0x7f11f40cdb90 msgr2=0x7f11f406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 shutdown_connections 2023-10-06T02:46:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:15.881+0000 7f11f910b700 1 -- 172.21.15.33:0/1266841979 wait complete. 2023-10-06T02:46:15.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:46:16.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.009+0000 7f71fda99700 1 Processor -- start 2023-10-06T02:46:16.016 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.009+0000 7f71fda99700 1 -- start start 2023-10-06T02:46:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71fda99700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71f80c5940 0x7f71f815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71fda99700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ca560 0x7f71f815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71fda99700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f806add0 con 0x7f71f80ade00 2023-10-06T02:46:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71fda99700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f80c80c0 con 0x7f71f80c5940 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71fda99700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80ae1d0 con 0x7f71f80ca560 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71f80c5940 0x7f71f815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f71f80c5940 0x7f71f815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53326/0 (socket says 172.21.15.33:53326) 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 -- 172.21.15.33:0/2127995765 learned_addr learned my addr 172.21.15.33:0/2127995765 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71fd298700 1 --2- 172.21.15.33:0/2127995765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ca560 0x7f71f815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 -- 172.21.15.33:0/2127995765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ca560 msgr2=0x7f71f815fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 --2- 172.21.15.33:0/2127995765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ca560 0x7f71f815fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 -- 172.21.15.33:0/2127995765 >> v1:172.21.15.167:6789/0 conn(0x7f71f80ade00 legacy=0x7f71f80c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.019 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 -- 172.21.15.33:0/2127995765 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f806a1c0 con 0x7f71f80c5940 2023-10-06T02:46:16.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f7fff700 1 --2- 172.21.15.33:0/2127995765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f815d4d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f71e40099b0 tx=0x7f71e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a64df389f9db75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:16.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f77fe700 1 -- 172.21.15.33:0/2127995765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400b860 con 0x7f71f80c5940 2023-10-06T02:46:16.020 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f77fe700 1 -- 172.21.15.33:0/2127995765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e400b9c0 con 0x7f71f80c5940 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.013+0000 7f71f77fe700 1 -- 172.21.15.33:0/2127995765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400bd60 con 0x7f71f80c5940 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 -- 172.21.15.33:0/2127995765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 msgr2=0x7f71f815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 --2- 172.21.15.33:0/2127995765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f815d4d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f71e40099b0 tx=0x7f71e4009250 comp rx=0 tx=0).stop 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 -- 172.21.15.33:0/2127995765 shutdown_connections 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 --2- 172.21.15.33:0/2127995765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ca560 0x7f71f815fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 --2- 172.21.15.33:0/2127995765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f815d4d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 -- 172.21.15.33:0/2127995765 >> 172.21.15.33:0/2127995765 conn(0x7f71f80c8240 msgr2=0x7f71f80c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 -- 172.21.15.33:0/2127995765 shutdown_connections 2023-10-06T02:46:16.021 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 -- 172.21.15.33:0/2127995765 wait complete. 2023-10-06T02:46:16.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 Processor -- start 2023-10-06T02:46:16.023 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.017+0000 7f71fda99700 1 -- start start 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ade00 0x7f71f80c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f814bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f80ca560 0x7f71f8158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8157f90 con 0x7f71f80c5940 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f814a6f0 con 0x7f71f80ca560 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f814a9d0 con 0x7f71f80ade00 2023-10-06T02:46:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f814bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f80ca560 0x7f71f8158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fca97700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ade00 0x7f71f80c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f814bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53334/0 (socket says 172.21.15.33:53334) 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f7fff700 1 -- 172.21.15.33:0/2608065207 learned_addr learned my addr 172.21.15.33:0/2608065207 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 -- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ade00 msgr2=0x7f71f80c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ade00 0x7f71f80c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 -- 172.21.15.33:0/2608065207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 msgr2=0x7f71f814bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f814bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 -- 172.21.15.33:0/2608065207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40094f0 con 0x7f71f80ca560 2023-10-06T02:46:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f7fff700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fd298700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f80ca560 0x7f71f8158420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71f0007ed0 tx=0x7f71f0009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0018070 con 0x7f71f80ca560 2023-10-06T02:46:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f8157c20 con 0x7f71f80ca560 2023-10-06T02:46:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f814b2b0 con 0x7f71f80ca560 2023-10-06T02:46:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f00099b0 con 0x7f71f80ca560 2023-10-06T02:46:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.021+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f000eda0 con 0x7f71f80ca560 2023-10-06T02:46:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.025+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d80052b0 con 0x7f71f80ca560 2023-10-06T02:46:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.025+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f71f0003020 con 0x7f71f80ca560 2023-10-06T02:46:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.025+0000 7f71f4ff9700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f71dc060550 0x7f71dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.025+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f71f007d250 con 0x7f71f80ca560 2023-10-06T02:46:16.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.025+0000 7f71fca97700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f71dc060550 0x7f71dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.029+0000 7f71fca97700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f71dc060550 0x7f71dc062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0039e0 tx=0x7f71ec007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.037+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f71f004d310 con 0x7f71f80ca560 2023-10-06T02:46:16.239 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.233+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f71d8005830 con 0x7f71f80ca560 2023-10-06T02:46:16.241 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T02:46:16.237+0000 7f7675d7e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:46:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71f4ff9700 1 -- 172.21.15.33:0/2608065207 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f71f0050930 con 0x7f71f80ca560 2023-10-06T02:46:16.253 INFO:tasks.ceph.mds.c.smithi033.stderr: -3288> 2023-10-06T02:45:28.761+0000 7f3acdbea700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:46:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f71dc060550 msgr2=0x7f71dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f71dc060550 0x7f71dc062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0039e0 tx=0x7f71ec007ec0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f80ca560 msgr2=0x7f71f8158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f80ca560 0x7f71f8158420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71f0007ed0 tx=0x7f71f0009850 comp rx=0 tx=0).stop 2023-10-06T02:46:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 shutdown_connections 2023-10-06T02:46:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f80ca560 0x7f71f8158420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71f80c5940 0x7f71f814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f71dc060550 0x7f71dc062a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 --2- 172.21.15.33:0/2608065207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71f80ade00 0x7f71f80c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 >> 172.21.15.33:0/2608065207 conn(0x7f71f80c8240 msgr2=0x7f71f80c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 shutdown_connections 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.249+0000 7f71fda99700 1 -- 172.21.15.33:0/2608065207 wait complete. 2023-10-06T02:46:16.256 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:46:16.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:46:16.281 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T02:46:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 Processor -- start 2023-10-06T02:46:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- start start 2023-10-06T02:46:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb8406b890 0x7feb840c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feb840ca640 0x7feb8415c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb84066fb0 con 0x7feb8415ccd0 2023-10-06T02:46:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb841536f0 con 0x7feb8406b890 2023-10-06T02:46:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb840cad90 con 0x7feb840ca640 2023-10-06T02:46:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feb8406b890 0x7feb840c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb83fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7feb8415ccd0 0x7feb84159190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60462/0 (socket says 172.21.15.33:60462) 2023-10-06T02:46:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb83fff700 1 -- 172.21.15.33:0/4188714214 learned_addr learned my addr 172.21.15.33:0/4188714214 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb7affd700 1 --2- 172.21.15.33:0/4188714214 >> v2:172.21.15.167:3300/0 conn(0x7feb840ca640 0x7feb8415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb837fe700 1 -- 172.21.15.33:0/4188714214 >> v2:172.21.15.167:3300/0 conn(0x7feb840ca640 msgr2=0x7feb8415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb837fe700 1 --2- 172.21.15.33:0/4188714214 >> v2:172.21.15.167:3300/0 conn(0x7feb840ca640 0x7feb8415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb82ffd700 1 -- 172.21.15.33:0/4188714214 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933693010 0 0) 0x7feb84066fb0 con 0x7feb8415ccd0 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb837fe700 1 -- 172.21.15.33:0/4188714214 >> v1:172.21.15.33:6789/0 conn(0x7feb8415ccd0 legacy=0x7feb84159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb837fe700 1 -- 172.21.15.33:0/4188714214 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8415d0d0 con 0x7feb8406b890 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb837fe700 1 --2- 172.21.15.33:0/4188714214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8406b890 0x7feb840c84c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feb84068b40 tx=0x7feb740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b374e7d8b271d4ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb82ffd700 1 -- 172.21.15.33:0/4188714214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400b6b0 con 0x7feb8406b890 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb82ffd700 1 -- 172.21.15.33:0/4188714214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7400b810 con 0x7feb8406b890 2023-10-06T02:46:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb82ffd700 1 -- 172.21.15.33:0/4188714214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400bbb0 con 0x7feb8406b890 2023-10-06T02:46:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- 172.21.15.33:0/4188714214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8406b890 msgr2=0x7feb840c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 --2- 172.21.15.33:0/4188714214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8406b890 0x7feb840c84c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feb84068b40 tx=0x7feb740091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- 172.21.15.33:0/4188714214 shutdown_connections 2023-10-06T02:46:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 --2- 172.21.15.33:0/4188714214 >> v2:172.21.15.167:3300/0 conn(0x7feb840ca640 0x7feb8415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 --2- 172.21.15.33:0/4188714214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8406b890 0x7feb840c84c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- 172.21.15.33:0/4188714214 >> 172.21.15.33:0/4188714214 conn(0x7feb840c8ad0 msgr2=0x7feb840c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- 172.21.15.33:0/4188714214 shutdown_connections 2023-10-06T02:46:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.369+0000 7feb88b11700 1 -- 172.21.15.33:0/4188714214 wait complete. 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 Processor -- start 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 -- start start 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb840ca640 0x7feb840c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8415ccd0 0x7feb84144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 0x7feb841560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841450a0 con 0x7feb84154890 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb84155c10 con 0x7feb8415ccd0 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb84154d30 con 0x7feb840ca640 2023-10-06T02:46:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 0x7feb841560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 0x7feb841560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53368/0 (socket says 172.21.15.33:53368) 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 -- 172.21.15.33:0/4167679102 learned_addr learned my addr 172.21.15.33:0/4167679102 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb837fe700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb840ca640 0x7feb840c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb7affd700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8415ccd0 0x7feb84144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 -- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb840ca640 msgr2=0x7feb840c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb840ca640 0x7feb840c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.380 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 -- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8415ccd0 msgr2=0x7feb84144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8415ccd0 0x7feb84144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 -- 172.21.15.33:0/4167679102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb74009420 con 0x7feb84154890 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb83fff700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 0x7feb841560a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00ccc0 tx=0x7feb7c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c00ab30 con 0x7feb84154890 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c00ac90 con 0x7feb84154890 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb84155000 con 0x7feb84154890 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c007340 con 0x7feb84154890 2023-10-06T02:46:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.373+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb841555c0 con 0x7feb84154890 2023-10-06T02:46:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.377+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb84155da0 con 0x7feb84154890 2023-10-06T02:46:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.377+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7feb7c0076b0 con 0x7feb84154890 2023-10-06T02:46:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.377+0000 7feb7bfff700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feb64060550 0x7feb64062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.377+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6953+0+0 (secure 0 0 0) 0x7feb7c07d210 con 0x7feb84154890 2023-10-06T02:46:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.377+0000 7feb837fe700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feb64060550 0x7feb64062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.377+0000 7feb837fe700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feb64060550 0x7feb64062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb840b2130 tx=0x7feb74017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.389+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feb7c04d300 con 0x7feb84154890 2023-10-06T02:46:16.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:16.581+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feb84071040 con 0x7feb84154890 2023-10-06T02:46:17.252 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.249+0000 7feb7bfff700 1 -- 172.21.15.33:0/4167679102 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7feb7c050920 con 0x7feb84154890 2023-10-06T02:46:17.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feb64060550 msgr2=0x7feb64062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feb64060550 0x7feb64062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb840b2130 tx=0x7feb74017000 comp rx=0 tx=0).stop 2023-10-06T02:46:17.257 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 msgr2=0x7feb841560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 0x7feb841560a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00ccc0 tx=0x7feb7c001a80 comp rx=0 tx=0).stop 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 shutdown_connections 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb84154890 0x7feb841560a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8415ccd0 0x7feb84144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feb64060550 0x7feb64062a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 --2- 172.21.15.33:0/4167679102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb840ca640 0x7feb840c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 >> 172.21.15.33:0/4167679102 conn(0x7feb840c8ad0 msgr2=0x7feb840c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 shutdown_connections 2023-10-06T02:46:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.253+0000 7feb88b11700 1 -- 172.21.15.33:0/4167679102 wait complete. 2023-10-06T02:46:17.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:46:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffbfff700 1 Processor -- start 2023-10-06T02:46:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffbfff700 1 -- start start 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc15af10 con 0x7f1ffc1593c0 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc159fe0 con 0x7f1ffc158ab0 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15a8c0 con 0x7f1ffc068ba0 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1ffc1593c0 0x7f1ffc15b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60478/0 (socket says 172.21.15.33:60478) 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffb7fe700 1 -- 172.21.15.33:0/2747869854 learned_addr learned my addr 172.21.15.33:0/2747869854 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffa7fc700 1 --2- 172.21.15.33:0/2747869854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463510236 0 0) 0x7f1ffc15af10 con 0x7f1ffc1593c0 2023-10-06T02:46:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8003570 con 0x7f1ffc1593c0 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478698744 0 0) 0x7f1ffc159fe0 con 0x7f1ffc158ab0 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ffc15af10 con 0x7f1ffc158ab0 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffa7fc700 1 -- 172.21.15.33:0/2747869854 >> v1:172.21.15.167:6789/0 conn(0x7f1ffc158ab0 legacy=0x7f1ffc068490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92341711 0 0) 0x7f1fe8003570 con 0x7f1ffc1593c0 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffa7fc700 1 -- 172.21.15.33:0/2747869854 >> v1:172.21.15.33:6789/0 conn(0x7f1ffc1593c0 legacy=0x7f1ffc15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.393+0000 7f1ffa7fc700 1 -- 172.21.15.33:0/2747869854 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc15aba0 con 0x7f1ffc068ba0 2023-10-06T02:46:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffa7fc700 1 --2- 172.21.15.33:0/2747869854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc158e80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009e90 tx=0x7f1fec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc26ba28b47964a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00e070 con 0x7f1ffc068ba0 2023-10-06T02:46:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec00b870 con 0x7f1ffc068ba0 2023-10-06T02:46:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/2747869854 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4001470 con 0x7f1ffc068ba0 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2747869854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 msgr2=0x7f1ffc158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2747869854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc158e80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009e90 tx=0x7f1fec009780 comp rx=0 tx=0).stop 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2747869854 shutdown_connections 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2747869854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc158e80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2747869854 >> 172.21.15.33:0/2747869854 conn(0x7f1ffc067a80 msgr2=0x7f1ffc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2747869854 shutdown_connections 2023-10-06T02:46:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.397+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2747869854 wait complete. 2023-10-06T02:46:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 Processor -- start 2023-10-06T02:46:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 -- start start 2023-10-06T02:46:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc158ab0 0x7f1ffc1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 0x7f1ffc157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc169270 con 0x7f1ffc1593c0 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc169aa0 con 0x7f1ffc158ab0 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc16d810 con 0x7f1ffc068ba0 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 0x7f1ffc157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 0x7f1ffc157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53380/0 (socket says 172.21.15.33:53380) 2023-10-06T02:46:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 -- 172.21.15.33:0/2346022085 learned_addr learned my addr 172.21.15.33:0/2346022085 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffa7fc700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc158ab0 0x7f1ffc1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffaffd700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 -- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 msgr2=0x7f1ffc14a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc14a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 -- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc158ab0 msgr2=0x7f1ffc1575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc158ab0 0x7f1ffc1575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.401+0000 7f1ffb7fe700 1 -- 172.21.15.33:0/2346022085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec009a20 con 0x7f1ffc1593c0 2023-10-06T02:46:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ffb7fe700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 0x7f1ffc157af0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c840 tx=0x7f1ff4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4007360 con 0x7f1ffc1593c0 2023-10-06T02:46:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff40074e0 con 0x7f1ffc1593c0 2023-10-06T02:46:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc16d530 con 0x7f1ffc1593c0 2023-10-06T02:46:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4004830 con 0x7f1ffc1593c0 2023-10-06T02:46:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc169730 con 0x7f1ffc1593c0 2023-10-06T02:46:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc071040 con 0x7f1ffc1593c0 2023-10-06T02:46:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1ff4004c00 con 0x7f1ffc1593c0 2023-10-06T02:46:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.405+0000 7f1ff2ffd700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fdc060550 0x7f1fdc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.409+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6953+0+0 (secure 0 0 0) 0x7f1ff407cb50 con 0x7f1ffc1593c0 2023-10-06T02:46:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.409+0000 7f1ffaffd700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fdc060550 0x7f1fdc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:17.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.409+0000 7f1ffaffd700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fdc060550 0x7f1fdc062a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0ce590 tx=0x7f1fe4007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.417+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ff404ccf0 con 0x7f1ffc1593c0 2023-10-06T02:46:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:17.581+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ffc0693c0 con 0x7f1ffc1593c0 2023-10-06T02:46:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.449+0000 7f1ff2ffd700 1 -- 172.21.15.33:0/2346022085 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1ff4050310 con 0x7f1ffc1593c0 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fdc060550 msgr2=0x7f1fdc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fdc060550 0x7f1fdc062a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0ce590 tx=0x7f1fe4007d50 comp rx=0 tx=0).stop 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 msgr2=0x7f1ffc157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 0x7f1ffc157af0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c840 tx=0x7f1ff4001980 comp rx=0 tx=0).stop 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 shutdown_connections 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ffc1593c0 0x7f1ffc157af0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc158ab0 0x7f1ffc1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fdc060550 0x7f1fdc062a00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 --2- 172.21.15.33:0/2346022085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1ffc068ba0 0x7f1ffc14a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 >> 172.21.15.33:0/2346022085 conn(0x7f1ffc067a80 msgr2=0x7f1ffc0c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 shutdown_connections 2023-10-06T02:46:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.457+0000 7f1ffbfff700 1 -- 172.21.15.33:0/2346022085 wait complete. 2023-10-06T02:46:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:46:18.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:46:18.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.581+0000 7fed0042b700 1 Processor -- start 2023-10-06T02:46:18.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.581+0000 7fed0042b700 1 -- start start 2023-10-06T02:46:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fed0042b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fecf80676c0 0x7fecf80af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fed0042b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf80cfa00 con 0x7fecf8158e70 2023-10-06T02:46:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fed0042b700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf80b3a50 con 0x7fecf80aeae0 2023-10-06T02:46:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fed0042b700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80b2180 con 0x7fecf80676c0 2023-10-06T02:46:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe1c7700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fecf8158e70 0x7fecf8066fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60484/0 (socket says 172.21.15.33:60484) 2023-10-06T02:46:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe1c7700 1 -- 172.21.15.33:0/899751491 learned_addr learned my addr 172.21.15.33:0/899751491 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd9c6700 1 --2- 172.21.15.33:0/899751491 >> v2:172.21.15.167:3300/0 conn(0x7fecf80676c0 0x7fecf80af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928787697 0 0) 0x7fecf80cfa00 con 0x7fecf8158e70 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece0003570 con 0x7fecf8158e70 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522404966 0 0) 0x7fece0003570 con 0x7fecf8158e70 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 >> v2:172.21.15.167:3300/0 conn(0x7fecf80676c0 msgr2=0x7fecf80af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 --2- 172.21.15.33:0/899751491 >> v2:172.21.15.167:3300/0 conn(0x7fecf80676c0 0x7fecf80af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 >> v1:172.21.15.167:6789/0 conn(0x7fecf80aeae0 legacy=0x7fecf80ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf80b2460 con 0x7fecf8158e70 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7fecf00017f0 con 0x7fecf8158e70 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 >> v1:172.21.15.33:6789/0 conn(0x7fecf8158e70 legacy=0x7fecf8066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece0003a60 0x7fece0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fece0004810 0x7fece0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece00051c0 0x7fece000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80b2180 con 0x7fece00051c0 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece000c160 con 0x7fece0003a60 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece000c320 con 0x7fece0004810 2023-10-06T02:46:18.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd9c6700 1 --2- 172.21.15.33:0/899751491 >> v2:172.21.15.167:3300/0 conn(0x7fecf80676c0 0x7fecf80af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:18.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece00051c0 0x7fece000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe1c7700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece0003a60 0x7fece0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd9c6700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fece0004810 0x7fece0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 -- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fece0004810 msgr2=0x7fece0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fece0004810 0x7fece0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 -- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece0003a60 msgr2=0x7fece0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece0003a60 0x7fece0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 -- 172.21.15.33:0/899751491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0003570 con 0x7fece00051c0 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe1c7700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece0003a60 0x7fece0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 -- 172.21.15.33:0/899751491 reap_dead start 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfe9c8700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece00051c0 0x7fece000ba60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fece80018b0 tx=0x7fece8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99be84dcef38a11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8012420 con 0x7fece00051c0 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece80125a0 con 0x7fece00051c0 2023-10-06T02:46:18.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.585+0000 7fecfd1c5700 1 -- 172.21.15.33:0/899751491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8012960 con 0x7fece00051c0 2023-10-06T02:46:18.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 -- 172.21.15.33:0/899751491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece00051c0 msgr2=0x7fece000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece00051c0 0x7fece000ba60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fece80018b0 tx=0x7fece8007d30 comp rx=0 tx=0).stop 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 -- 172.21.15.33:0/899751491 shutdown_connections 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 --2- 172.21.15.33:0/899751491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece00051c0 0x7fece000ba60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 -- 172.21.15.33:0/899751491 >> 172.21.15.33:0/899751491 conn(0x7fecf80691b0 msgr2=0x7fecf80b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 -- 172.21.15.33:0/899751491 shutdown_connections 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 -- 172.21.15.33:0/899751491 wait complete. 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 Processor -- start 2023-10-06T02:46:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.589+0000 7fed0042b700 1 -- start start 2023-10-06T02:46:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece00051c0 0x7fecf8159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 0x7fecf815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecf815be80 0x7fecf81696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf816a730 con 0x7fecf8151c70 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815c2f0 con 0x7fece00051c0 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815d320 con 0x7fecf815be80 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 0x7fecf815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 0x7fecf815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53408/0 (socket says 172.21.15.33:53408) 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 -- 172.21.15.33:0/3118293672 learned_addr learned my addr 172.21.15.33:0/3118293672 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfe1c7700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece00051c0 0x7fecf8159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfe9c8700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecf815be80 0x7fecf81696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 -- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecf815be80 msgr2=0x7fecf81696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecf815be80 0x7fecf81696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 -- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece00051c0 msgr2=0x7fecf8159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:18.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece00051c0 0x7fecf8159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:18.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 -- 172.21.15.33:0/3118293672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf816a450 con 0x7fecf8151c70 2023-10-06T02:46:18.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecfd9c6700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 0x7fecf815b940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fecec011d70 tx=0x7fecec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec011e70 con 0x7fecf8151c70 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec00fcd0 con 0x7fecf8151c70 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf815ccd0 con 0x7fecf8151c70 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec01d6d0 con 0x7fecf8151c70 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.593+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf8159780 con 0x7fecf8151c70 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.597+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fecec01dac0 con 0x7fecf8151c70 2023-10-06T02:46:18.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.597+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf815cb30 con 0x7fecf8151c70 2023-10-06T02:46:18.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.597+0000 7fecf67fc700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fece4060550 0x7fece4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.597+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6515+0+0 (secure 0 0 0) 0x7fecec018070 con 0x7fecf8151c70 2023-10-06T02:46:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.597+0000 7fecfe1c7700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fece4060550 0x7fece4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.597+0000 7fecfe1c7700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fece4060550 0x7fece4062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a0c0 tx=0x7fecf00021d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:18.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.609+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fecf815cb30 con 0x7fecf8151c70 2023-10-06T02:46:18.776 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:18.773+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecf80d0e00 con 0x7fecf8151c70 2023-10-06T02:46:19.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.461+0000 7fecf67fc700 1 -- 172.21.15.33:0/3118293672 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7fecf80d0e00 con 0x7fecf8151c70 2023-10-06T02:46:19.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fece4060550 msgr2=0x7fece4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:19.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fece4060550 0x7fece4062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a0c0 tx=0x7fecf00021d0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 msgr2=0x7fecf815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:19.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 0x7fecf815b940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fecec011d70 tx=0x7fecec0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 shutdown_connections 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fece4060550 0x7fece4062a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecf815be80 0x7fecf81696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecf8151c70 0x7fecf815b940 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 --2- 172.21.15.33:0/3118293672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fece00051c0 0x7fecf8159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 >> 172.21.15.33:0/3118293672 conn(0x7fecf80691b0 msgr2=0x7fecf80ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 shutdown_connections 2023-10-06T02:46:19.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.465+0000 7fed0042b700 1 -- 172.21.15.33:0/3118293672 wait complete. 2023-10-06T02:46:19.474 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T02:46:19.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-06T02:46:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.593+0000 7f4805428700 1 Processor -- start 2023-10-06T02:46:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.593+0000 7f4805428700 1 -- start start 2023-10-06T02:46:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4805428700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48000cac00 0x7f48000cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4805428700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48001588a0 0x7f4800158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4805428700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48000ce140 con 0x7f48000af280 2023-10-06T02:46:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4805428700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48000cb510 con 0x7f48000cac00 2023-10-06T02:46:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4805428700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4800068b30 con 0x7f48001588a0 2023-10-06T02:46:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47ff7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f48000af280 0x7f4800068420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60486/0 (socket says 172.21.15.33:60486) 2023-10-06T02:46:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47ff7fe700 1 -- 172.21.15.33:0/3157071212 learned_addr learned my addr 172.21.15.33:0/3157071212 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4804c27700 1 --2- 172.21.15.33:0/3157071212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48001588a0 0x7f4800158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47fffff700 1 --2- 172.21.15.33:0/3157071212 >> v2:172.21.15.167:3300/0 conn(0x7f48000cac00 0x7f48000cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47feffd700 1 -- 172.21.15.33:0/3157071212 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258644793 0 0) 0x7f48000ce140 con 0x7f48000af280 2023-10-06T02:46:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47feffd700 1 -- 172.21.15.33:0/3157071212 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47ec003570 con 0x7f48000af280 2023-10-06T02:46:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4804c27700 1 -- 172.21.15.33:0/3157071212 >> v2:172.21.15.167:3300/0 conn(0x7f48000cac00 msgr2=0x7f48000cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4804c27700 1 --2- 172.21.15.33:0/3157071212 >> v2:172.21.15.167:3300/0 conn(0x7f48000cac00 0x7f48000cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4804c27700 1 -- 172.21.15.33:0/3157071212 >> v1:172.21.15.33:6789/0 conn(0x7f48000af280 legacy=0x7f4800068420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4804c27700 1 -- 172.21.15.33:0/3157071212 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48000cd880 con 0x7f48001588a0 2023-10-06T02:46:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f4804c27700 1 --2- 172.21.15.33:0/3157071212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48001588a0 0x7f4800158ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009e60 tx=0x7f47f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e59fb3551c7048 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47feffd700 1 -- 172.21.15.33:0/3157071212 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800e070 con 0x7f48001588a0 2023-10-06T02:46:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47feffd700 1 -- 172.21.15.33:0/3157071212 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800bab0 con 0x7f48001588a0 2023-10-06T02:46:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.597+0000 7f47feffd700 1 -- 172.21.15.33:0/3157071212 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800be30 con 0x7f48001588a0 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 -- 172.21.15.33:0/3157071212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48001588a0 msgr2=0x7f4800158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 --2- 172.21.15.33:0/3157071212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48001588a0 0x7f4800158ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009e60 tx=0x7f47f8009780 comp rx=0 tx=0).stop 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 -- 172.21.15.33:0/3157071212 shutdown_connections 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 --2- 172.21.15.33:0/3157071212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48001588a0 0x7f4800158ce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 --2- 172.21.15.33:0/3157071212 >> v2:172.21.15.167:3300/0 conn(0x7f48000cac00 0x7f48000cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 -- 172.21.15.33:0/3157071212 >> 172.21.15.33:0/3157071212 conn(0x7f4800067a30 msgr2=0x7f48000cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 -- 172.21.15.33:0/3157071212 shutdown_connections 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 -- 172.21.15.33:0/3157071212 wait complete. 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 Processor -- start 2023-10-06T02:46:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.601+0000 7f4805428700 1 -- start start 2023-10-06T02:46:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 0x7f480015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48000cac00 0x7f48000b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48001588a0 0x7f480016cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480015fbd0 con 0x7f48000af280 2023-10-06T02:46:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48000b40f0 con 0x7f48001588a0 2023-10-06T02:46:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480016c620 con 0x7f48000cac00 2023-10-06T02:46:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 0x7f480015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:19.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 0x7f480015f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53418/0 (socket says 172.21.15.33:53418) 2023-10-06T02:46:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 -- 172.21.15.33:0/2199593428 learned_addr learned my addr 172.21.15.33:0/2199593428 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47ff7fe700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48000cac00 0x7f48000b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4804c27700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48001588a0 0x7f480016cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 -- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48000cac00 msgr2=0x7f48000b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48000cac00 0x7f48000b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 -- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48001588a0 msgr2=0x7f480016cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48001588a0 0x7f480016cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 -- 172.21.15.33:0/2199593428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8009a20 con 0x7f48000af280 2023-10-06T02:46:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47fffff700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 0x7f480015f530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47f000ec80 tx=0x7f47f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f000c850 con 0x7f48000af280 2023-10-06T02:46:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f000c9b0 con 0x7f48000af280 2023-10-06T02:46:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480015cbf0 con 0x7f48000af280 2023-10-06T02:46:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f000cd50 con 0x7f48000af280 2023-10-06T02:46:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.605+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480016c8c0 con 0x7f48000af280 2023-10-06T02:46:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.609+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4800071040 con 0x7f48000af280 2023-10-06T02:46:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.609+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f47f0013680 con 0x7f48000af280 2023-10-06T02:46:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.609+0000 7f47f7fff700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f47e0060550 0x7f47e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:19.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.609+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6043+0+0 (secure 0 0 0) 0x7f47f007cb70 con 0x7f48000af280 2023-10-06T02:46:19.620 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.609+0000 7f47ff7fe700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f47e0060550 0x7f47e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:19.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.609+0000 7f47ff7fe700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f47e0060550 0x7f47e0062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48000b3b20 tx=0x7f47e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:19.621 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.617+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f47f004d010 con 0x7f48000af280 2023-10-06T02:46:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:19.777+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48000cdaf0 con 0x7f48000af280 2023-10-06T02:46:19.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:46:19.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:46:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.469+0000 7f47f7fff700 1 -- 172.21.15.33:0/2199593428 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v36) v1 ==== 178+0+0 (secure 0 0 0) 0x7f47f0050630 con 0x7f48000af280 2023-10-06T02:46:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f47e0060550 msgr2=0x7f47e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f47e0060550 0x7f47e0062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48000b3b20 tx=0x7f47e8009040 comp rx=0 tx=0).stop 2023-10-06T02:46:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 msgr2=0x7f480015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 0x7f480015f530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47f000ec80 tx=0x7f47f00091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 shutdown_connections 2023-10-06T02:46:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f47e0060550 0x7f47e0062a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48000cac00 0x7f48000b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48001588a0 0x7f480016cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 --2- 172.21.15.33:0/2199593428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48000af280 0x7f480015f530 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 >> 172.21.15.33:0/2199593428 conn(0x7f4800067a30 msgr2=0x7f48000ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 shutdown_connections 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.473+0000 7f4805428700 1 -- 172.21.15.33:0/2199593428 wait complete. 2023-10-06T02:46:20.479 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' removed 2023-10-06T02:46:20.492 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T02:46:20.492 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T02:46:20.493 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:46:20.493 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:46:20.493 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T02:46:20.493 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T02:46:20.494 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:46:20.494 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:46:20.494 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T02:46:20.494 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T02:46:20.494 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:46:20.495 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:46:20.495 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T02:46:20.495 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T02:46:20.495 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:46:20.495 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:46:20.506 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:46:20.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T02:46:20.507 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:46:20.507 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T02:46:20.508 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:46:20.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T02:46:20.510 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:46:20.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T02:46:20.511 INFO:tasks.ceph.mds.a:Started 2023-10-06T02:46:20.511 DEBUG:teuthology.parallel:result is None 2023-10-06T02:46:20.512 INFO:tasks.ceph.mds.c:Started 2023-10-06T02:46:20.512 DEBUG:teuthology.parallel:result is None 2023-10-06T02:46:20.550 INFO:tasks.ceph.mds.b:Started 2023-10-06T02:46:20.551 DEBUG:teuthology.parallel:result is None 2023-10-06T02:46:20.551 INFO:tasks.ceph.mds.d:Started 2023-10-06T02:46:20.551 DEBUG:teuthology.parallel:result is None 2023-10-06T02:46:20.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T02:46:20.579 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2023-10-06T02:46:20.587 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T02:46:20.626 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2023-10-06T02:46:20.633 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2023-10-06T02:46:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.665+0000 7f787de1a700 1 Processor -- start 2023-10-06T02:46:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.665+0000 7f787de1a700 1 -- start start 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c0d10 0x7f78780c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780c99d0 con 0x7f78780c9550 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7878067050 con 0x7f7878160fd0 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7878067330 con 0x7f78780c0d10 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7876ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f78780c9550 0x7f78780680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60496/0 (socket says 172.21.15.33:60496) 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7876ffd700 1 -- 172.21.15.33:0/181152171 learned_addr learned my addr 172.21.15.33:0/181152171 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7877fff700 1 --2- 172.21.15.33:0/181152171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c0d10 0x7f78780c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311077023 0 0) 0x7f78780c99d0 con 0x7f78780c9550 2023-10-06T02:46:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7864003570 con 0x7f78780c9550 2023-10-06T02:46:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957196933 0 0) 0x7f7878067050 con 0x7f7878160fd0 2023-10-06T02:46:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78780c99d0 con 0x7f7878160fd0 2023-10-06T02:46:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7877fff700 1 -- 172.21.15.33:0/181152171 >> v1:172.21.15.167:6789/0 conn(0x7f7878160fd0 legacy=0x7f78781613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7877fff700 1 -- 172.21.15.33:0/181152171 >> v1:172.21.15.33:6789/0 conn(0x7f78780c9550 legacy=0x7f78780680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7877fff700 1 -- 172.21.15.33:0/181152171 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7878067610 con 0x7f78780c0d10 2023-10-06T02:46:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f7877fff700 1 --2- 172.21.15.33:0/181152171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c0d10 0x7f78780c1170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7868009910 tx=0x7f78680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32d3a88f00ad22d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868011630 con 0x7f78780c0d10 2023-10-06T02:46:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868011790 con 0x7f78780c0d10 2023-10-06T02:46:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f78767fc700 1 -- 172.21.15.33:0/181152171 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868011b30 con 0x7f78780c0d10 2023-10-06T02:46:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 -- 172.21.15.33:0/181152171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c0d10 msgr2=0x7f78780c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 --2- 172.21.15.33:0/181152171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c0d10 0x7f78780c1170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7868009910 tx=0x7f78680091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 -- 172.21.15.33:0/181152171 shutdown_connections 2023-10-06T02:46:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 --2- 172.21.15.33:0/181152171 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c0d10 0x7f78780c1170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.669+0000 7f787de1a700 1 -- 172.21.15.33:0/181152171 >> 172.21.15.33:0/181152171 conn(0x7f78780c83f0 msgr2=0x7f78780c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.673+0000 7f787de1a700 1 -- 172.21.15.33:0/181152171 shutdown_connections 2023-10-06T02:46:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.673+0000 7f787de1a700 1 -- 172.21.15.33:0/181152171 wait complete. 2023-10-06T02:46:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.673+0000 7f787de1a700 1 Processor -- start 2023-10-06T02:46:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.673+0000 7f787de1a700 1 -- start start 2023-10-06T02:46:20.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 0x7f7878157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:20.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c9550 0x7f78780c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:20.683 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7878160fd0 0x7f78780c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878162cb0 con 0x7f78780c0d10 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878162f90 con 0x7f7878160fd0 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7878163270 con 0x7f78780c9550 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 0x7f7878157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 0x7f7878157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53456/0 (socket says 172.21.15.33:53456) 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 -- 172.21.15.33:0/2475675 learned_addr learned my addr 172.21.15.33:0/2475675 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f7877fff700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7878160fd0 0x7f78780c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f7876ffd700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c9550 0x7f78780c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:20.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 -- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c9550 msgr2=0x7f78780c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c9550 0x7f78780c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 -- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7878160fd0 msgr2=0x7f78780c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7878160fd0 0x7f78780c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 -- 172.21.15.33:0/2475675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7868009420 con 0x7f78780c0d10 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f7877fff700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7878160fd0 0x7f78780c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f78777fe700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 0x7f7878157c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f787000c900 tx=0x7f7870002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000cea0 con 0x7f78780c0d10 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787000a870 con 0x7f78780c0d10 2023-10-06T02:46:20.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7878163550 con 0x7f78780c0d10 2023-10-06T02:46:20.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000ac30 con 0x7f78780c0d10 2023-10-06T02:46:20.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.677+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7878163d50 con 0x7f78780c0d10 2023-10-06T02:46:20.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.681+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7878163a50 con 0x7f78780c0d10 2023-10-06T02:46:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.681+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7870012630 con 0x7f78780c0d10 2023-10-06T02:46:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.681+0000 7f786f7fe700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7858060550 0x7f7858062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.681+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f787007ccb0 con 0x7f78780c0d10 2023-10-06T02:46:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.681+0000 7f7876ffd700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7858060550 0x7f7858062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.681+0000 7f7876ffd700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7858060550 0x7f7858062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f787806aa20 tx=0x7f7860009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:20.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.689+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f787004d260 con 0x7f78780c0d10 2023-10-06T02:46:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:20.857+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f78780710b0 con 0x7f78780c0d10 2023-10-06T02:46:21.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f786f7fe700 1 -- 172.21.15.33:0/2475675 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7870016080 con 0x7f78780c0d10 2023-10-06T02:46:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7858060550 msgr2=0x7f7858062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7858060550 0x7f7858062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f787806aa20 tx=0x7f7860009040 comp rx=0 tx=0).stop 2023-10-06T02:46:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 msgr2=0x7f7878157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 0x7f7878157c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f787000c900 tx=0x7f7870002780 comp rx=0 tx=0).stop 2023-10-06T02:46:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 shutdown_connections 2023-10-06T02:46:21.500 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7858060550 0x7f7858062a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78780c9550 0x7f78780c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7878160fd0 0x7f78780c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 --2- 172.21.15.33:0/2475675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78780c0d10 0x7f7878157c50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 >> 172.21.15.33:0/2475675 conn(0x7f78780c83f0 msgr2=0x7f78780691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 shutdown_connections 2023-10-06T02:46:21.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.493+0000 7f787de1a700 1 -- 172.21.15.33:0/2475675 wait complete. 2023-10-06T02:46:21.502 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2023-10-06T02:46:21.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T02:46:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 Processor -- start 2023-10-06T02:46:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- start start 2023-10-06T02:46:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f942c0c7450 0x7f942c067970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0c86b0 con 0x7f942c154c40 2023-10-06T02:46:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0c7880 con 0x7f942c0c7450 2023-10-06T02:46:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c7aa0 con 0x7f942c06e170 2023-10-06T02:46:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f942c0c7450 0x7f942c067970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f942c0c7450 0x7f942c067970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53472/0 (socket says 172.21.15.33:53472) 2023-10-06T02:46:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 -- 172.21.15.33:0/3286597735 learned_addr learned my addr 172.21.15.33:0/3286597735 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f9430fab700 1 --2- 172.21.15.33:0/3286597735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 -- 172.21.15.33:0/3286597735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 msgr2=0x7f942c06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 --2- 172.21.15.33:0/3286597735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c06e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 -- 172.21.15.33:0/3286597735 >> v1:172.21.15.167:6789/0 conn(0x7f942c154c40 legacy=0x7f942c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:21.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 -- 172.21.15.33:0/3286597735 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0c7d80 con 0x7f942c0c7450 2023-10-06T02:46:21.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942bfff700 1 --2- 172.21.15.33:0/3286597735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c0c7450 0x7f942c067970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9424009f20 tx=0x7f9424009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a88646ef6d231f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:21.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942b7fe700 1 -- 172.21.15.33:0/3286597735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400e070 con 0x7f942c0c7450 2023-10-06T02:46:21.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942b7fe700 1 -- 172.21.15.33:0/3286597735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942400ba40 con 0x7f942c0c7450 2023-10-06T02:46:21.679 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f942b7fe700 1 -- 172.21.15.33:0/3286597735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240092e0 con 0x7f942c0c7450 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- 172.21.15.33:0/3286597735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c0c7450 msgr2=0x7f942c067970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 --2- 172.21.15.33:0/3286597735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c0c7450 0x7f942c067970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9424009f20 tx=0x7f9424009840 comp rx=0 tx=0).stop 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- 172.21.15.33:0/3286597735 shutdown_connections 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 --2- 172.21.15.33:0/3286597735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c0c7450 0x7f942c067970 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 --2- 172.21.15.33:0/3286597735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c06e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- 172.21.15.33:0/3286597735 >> 172.21.15.33:0/3286597735 conn(0x7f942c06aa50 msgr2=0x7f942c0cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- 172.21.15.33:0/3286597735 shutdown_connections 2023-10-06T02:46:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.669+0000 7f943320f700 1 -- 172.21.15.33:0/3286597735 wait complete. 2023-10-06T02:46:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 Processor -- start 2023-10-06T02:46:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 -- start start 2023-10-06T02:46:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c0c7450 0x7f942c16a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 0x7f942c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c5c70 con 0x7f942c154c40 2023-10-06T02:46:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c16ad20 con 0x7f942c0c7450 2023-10-06T02:46:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f943320f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c155bb0 con 0x7f942c06e170 2023-10-06T02:46:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 0x7f942c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 0x7f942c1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53478/0 (socket says 172.21.15.33:53478) 2023-10-06T02:46:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 -- 172.21.15.33:0/4182241300 learned_addr learned my addr 172.21.15.33:0/4182241300 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f9430fab700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f942bfff700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c0c7450 0x7f942c16a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 -- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 msgr2=0x7f942c0c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c0c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 -- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c0c7450 msgr2=0x7f942c16a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c0c7450 0x7f942c16a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f94317ac700 1 -- 172.21.15.33:0/4182241300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009ae0 con 0x7f942c154c40 2023-10-06T02:46:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.673+0000 7f942bfff700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c0c7450 0x7f942c16a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f94317ac700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 0x7f942c1584d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b510 tx=0x7f941c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00f510 con 0x7f942c154c40 2023-10-06T02:46:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c00f670 con 0x7f942c154c40 2023-10-06T02:46:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c0c4bd0 con 0x7f942c154c40 2023-10-06T02:46:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00fa30 con 0x7f942c154c40 2023-10-06T02:46:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c0c5470 con 0x7f942c154c40 2023-10-06T02:46:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f941c01a400 con 0x7f942c154c40 2023-10-06T02:46:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c0c7090 con 0x7f942c154c40 2023-10-06T02:46:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9428ff9700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9410060550 0x7f9410062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f941c07c110 con 0x7f942c154c40 2023-10-06T02:46:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.677+0000 7f9430fab700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9410060550 0x7f9410062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.681+0000 7f9430fab700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9410060550 0x7f9410062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9414006fd0 tx=0x7f9414008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.685+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f942c0c7090 con 0x7f942c154c40 2023-10-06T02:46:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.905+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f942c0c7090 con 0x7f942c154c40 2023-10-06T02:46:21.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.905+0000 7f9428ff9700 1 -- 172.21.15.33:0/4182241300 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f942c0c7090 con 0x7f942c154c40 2023-10-06T02:46:21.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:21.910 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:46:21.911 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2023-10-06T02:46:21.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.911 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2023-10-06T02:46:21.911 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9lmIpTKxAAaEJIguG5VOzRyEvP9O380Q==", 2023-10-06T02:46:21.911 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.912 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.912 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T02:46:21.912 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T02:46:21.912 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9lwO51NRAA4LpEWhRzJs7uOlPCN/qnmw==", 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T02:46:21.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9luwLtLhAAqMTGDQaDKQ3Km6+zJNQFSA==", 2023-10-06T02:46:21.914 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T02:46:21.915 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T02:46:21.915 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.916 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2023-10-06T02:46:21.916 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9laNKfOBAAhS+9+Pn1wJFPz/vX6dZI/A==", 2023-10-06T02:46:21.916 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.916 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCHdB9lHll+LRAAFEJKO2bT5nUkjn39YbcUPg==", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.917 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCLdB9lgBcyBRAAXbz0lU4ud1hBRoJ2iwO0Lg==", 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCOdB9lUp9PChAAKWdNaHeOJKyGp81y+QRs1Q==", 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2023-10-06T02:46:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCRdB9ll6JZDBAAsnkvAAbpz3fQM4dJSqJrIw==", 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCWdB9lWVnKLBAAtCoJpFucjfywP/S7yJNalw==", 2023-10-06T02:46:21.920 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCadB9la1/mBxAAdmk6mMu3x9eugP16xIakpA==", 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCddB9l3XIaDhAAQbwhnmEiwsQSXl37930UAQ==", 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.922 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCgdB9l7y7kExAA3qsifJxMWRbTTRvY1Jqclg==", 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2023-10-06T02:46:21.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCFdB9lylEVABAAyiDWlhvhDg77IFhO/oclZQ==", 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T02:46:21.924 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCFdB9lrA+ZCBAAwUmcQFkwJkECp56B5mJ/cw==", 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T02:46:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCFdB9lXmNuBRAATJNuIDqIgXyJqPPlW6Gcng==", 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T02:46:21.926 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCFdB9l4KF/CxAAE3gDNFao3CXhSdzYQ+8pEQ==", 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2023-10-06T02:46:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9l+3Z7GxAASlr7hRSy4XDknvDXsutkAQ==", 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2023-10-06T02:46:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mds", 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCqdB9lKwgHFRAAE0pm86MQxjE0NO6esD0Jvg==", 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCqdB9laSMHFRAAzqxPua77Rcr1y6YcXRwJ6A==", 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-osd", 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCqdB9lBDoHFRAAlcwJCkVPbhNMNMZTZSroRA==", 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCqdB9lIFAHFRAAWhPB/M8EVjGXhc+eJVOVvQ==", 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCqdB9l42YHFRAAZqxlP4xBx2a/G2rNXuAyCw==", 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCqdB9lpIAHFRAAvgnmL2/L/UBoRWAlNQEMYw==", 2023-10-06T02:46:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9lLk2DKBAAwraSyfivgdCX6JLtapLX8w==", 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQCEdB9l66GOJRAA5/Gl0H9QU0iET4rMvex3CQ==", 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2023-10-06T02:46:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2023-10-06T02:46:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2023-10-06T02:46:21.935 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:46:21.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9410060550 msgr2=0x7f9410062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9410060550 0x7f9410062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9414006fd0 tx=0x7f9414008040 comp rx=0 tx=0).stop 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 msgr2=0x7f942c1584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 0x7f942c1584d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b510 tx=0x7f941c006f90 comp rx=0 tx=0).stop 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 shutdown_connections 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f942c154c40 0x7f942c1584d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f942c0c7450 0x7f942c16a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9410060550 0x7f9410062a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 --2- 172.21.15.33:0/4182241300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f942c06e170 0x7f942c0c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 >> 172.21.15.33:0/4182241300 conn(0x7f942c06aa50 msgr2=0x7f942c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:21.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 shutdown_connections 2023-10-06T02:46:21.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:21.909+0000 7f943320f700 1 -- 172.21.15.33:0/4182241300 wait complete. 2023-10-06T02:46:21.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T02:46:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.037+0000 7fc610160700 1 Processor -- start 2023-10-06T02:46:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.037+0000 7fc610160700 1 -- start start 2023-10-06T02:46:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6080cc350 0x7fc608156060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc6080c8e50 0x7fc6080680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc60806bfc0 0x7fc60806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080c8b50 con 0x7fc60806bfc0 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60806c960 con 0x7fc6080c8e50 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc608159790 con 0x7fc6080cc350 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc60806bfc0 0x7fc60806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc60806bfc0 0x7fc60806c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53482/0 (socket says 172.21.15.33:53482) 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 -- 172.21.15.33:0/2552344277 learned_addr learned my addr 172.21.15.33:0/2552344277 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60defc700 1 --2- 172.21.15.33:0/2552344277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6080cc350 0x7fc608156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60d6fb700 1 --2- 172.21.15.33:0/2552344277 >> v2:172.21.15.167:3300/0 conn(0x7fc6080c8e50 0x7fc6080680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 -- 172.21.15.33:0/2552344277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6080cc350 msgr2=0x7fc608156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 --2- 172.21.15.33:0/2552344277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6080cc350 0x7fc608156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 -- 172.21.15.33:0/2552344277 >> v2:172.21.15.167:3300/0 conn(0x7fc6080c8e50 msgr2=0x7fc6080680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 --2- 172.21.15.33:0/2552344277 >> v2:172.21.15.167:3300/0 conn(0x7fc6080c8e50 0x7fc6080680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 -- 172.21.15.33:0/2552344277 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6081566b0 con 0x7fc60806bfc0 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60e6fd700 1 --2- 172.21.15.33:0/2552344277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc60806bfc0 0x7fc60806c420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ca570 tx=0x7fc600009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e3dca965caf719 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60cefa700 1 -- 172.21.15.33:0/2552344277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60000e070 con 0x7fc60806bfc0 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60cefa700 1 -- 172.21.15.33:0/2552344277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60000b970 con 0x7fc60806bfc0 2023-10-06T02:46:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc60cefa700 1 -- 172.21.15.33:0/2552344277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000092e0 con 0x7fc60806bfc0 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- 172.21.15.33:0/2552344277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc60806bfc0 msgr2=0x7fc60806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- 172.21.15.33:0/2552344277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc60806bfc0 0x7fc60806c420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ca570 tx=0x7fc600009840 comp rx=0 tx=0).stop 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- 172.21.15.33:0/2552344277 shutdown_connections 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- 172.21.15.33:0/2552344277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc60806bfc0 0x7fc60806c420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- 172.21.15.33:0/2552344277 >> v2:172.21.15.167:3300/0 conn(0x7fc6080c8e50 0x7fc6080680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 --2- 172.21.15.33:0/2552344277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc6080cc350 0x7fc608156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- 172.21.15.33:0/2552344277 >> 172.21.15.33:0/2552344277 conn(0x7fc60806e7c0 msgr2=0x7fc60806ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- 172.21.15.33:0/2552344277 shutdown_connections 2023-10-06T02:46:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.041+0000 7fc610160700 1 -- 172.21.15.33:0/2552344277 wait complete. 2023-10-06T02:46:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 Processor -- start 2023-10-06T02:46:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 -- start start 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc60806bfc0 0x7fc6081555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6080c8e50 0x7fc6080ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 0x7fc6080ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080ae3f0 con 0x7fc6080cc350 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60816a4c0 con 0x7fc6080c8e50 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc610160700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60816a170 con 0x7fc60806bfc0 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 0x7fc6080ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 0x7fc6080ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53498/0 (socket says 172.21.15.33:53498) 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 -- 172.21.15.33:0/3989540592 learned_addr learned my addr 172.21.15.33:0/3989540592 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60defc700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc60806bfc0 0x7fc6081555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60d6fb700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6080c8e50 0x7fc6080ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 -- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc60806bfc0 msgr2=0x7fc6081555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc60806bfc0 0x7fc6081555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 -- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6080c8e50 msgr2=0x7fc6080ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6080c8e50 0x7fc6080ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60e6fd700 1 -- 172.21.15.33:0/3989540592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600009ae0 con 0x7fc6080cc350 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60defc700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc60806bfc0 0x7fc6081555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.045+0000 7fc60d6fb700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6080c8e50 0x7fc6080ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc60e6fd700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 0x7fc6080ac800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc600009880 tx=0x7fc600018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600008bc0 con 0x7fc6080cc350 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600008d40 con 0x7fc6080cc350 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000266b0 con 0x7fc6080cc350 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6080ace80 con 0x7fc6080cc350 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6080ad440 con 0x7fc6080cc350 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60816a670 con 0x7fc6080cc350 2023-10-06T02:46:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc600003040 con 0x7fc6080cc350 2023-10-06T02:46:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc5fdffb700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc5ec060550 0x7fc5ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fc60007e010 con 0x7fc6080cc350 2023-10-06T02:46:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc60defc700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc5ec060550 0x7fc5ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.049+0000 7fc60defc700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc5ec060550 0x7fc5ec062a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc604009ce0 tx=0x7fc604008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.057+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc60816a670 con 0x7fc6080cc350 2023-10-06T02:46:22.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.277+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc60816a630 con 0x7fc6080cc350 2023-10-06T02:46:22.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc5fdffb700 1 -- 172.21.15.33:0/3989540592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc60004e760 con 0x7fc6080cc350 2023-10-06T02:46:22.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc5ec060550 msgr2=0x7fc5ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc5ec060550 0x7fc5ec062a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc604009ce0 tx=0x7fc604008040 comp rx=0 tx=0).stop 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 msgr2=0x7fc6080ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 0x7fc6080ac800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc600009880 tx=0x7fc600018000 comp rx=0 tx=0).stop 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 shutdown_connections 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6080cc350 0x7fc6080ac800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6080c8e50 0x7fc6080ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc5ec060550 0x7fc5ec062a00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 --2- 172.21.15.33:0/3989540592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc60806bfc0 0x7fc6081555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 >> 172.21.15.33:0/3989540592 conn(0x7fc60806e7c0 msgr2=0x7fc6080c1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 shutdown_connections 2023-10-06T02:46:22.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.285+0000 7fc610160700 1 -- 172.21.15.33:0/3989540592 wait complete. 2023-10-06T02:46:22.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T02:46:22.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 Processor -- start 2023-10-06T02:46:22.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 -- start start 2023-10-06T02:46:22.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158bf0 0x7fd1ac0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1ac0b0530 0x7fd1ac0b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0b2610 con 0x7fd1ac158fc0 2023-10-06T02:46:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac067c10 con 0x7fd1ac0b0530 2023-10-06T02:46:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1b11d8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15d830 con 0x7fd1ac158bf0 2023-10-06T02:46:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1ac0b0530 0x7fd1ac0b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1ac0b0530 0x7fd1ac0b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53512/0 (socket says 172.21.15.33:53512) 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 -- 172.21.15.33:0/209685139 learned_addr learned my addr 172.21.15.33:0/209685139 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1aad9d700 1 --2- 172.21.15.33:0/209685139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158bf0 0x7fd1ac0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 -- 172.21.15.33:0/209685139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158bf0 msgr2=0x7fd1ac0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 --2- 172.21.15.33:0/209685139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158bf0 0x7fd1ac0b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 -- 172.21.15.33:0/209685139 >> v1:172.21.15.167:6789/0 conn(0x7fd1ac158fc0 legacy=0x7fd1ac15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 -- 172.21.15.33:0/209685139 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0b00a0 con 0x7fd1ac0b0530 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.413+0000 7fd1a259c700 1 --2- 172.21.15.33:0/209685139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac0b0530 0x7fd1ac0b20d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1940099b0 tx=0x7fd194009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72bf1c2cf6e28fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1aa59c700 1 -- 172.21.15.33:0/209685139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194009ea0 con 0x7fd1ac0b0530 2023-10-06T02:46:22.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1aa59c700 1 -- 172.21.15.33:0/209685139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400b860 con 0x7fd1ac0b0530 2023-10-06T02:46:22.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1aa59c700 1 -- 172.21.15.33:0/209685139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400bc00 con 0x7fd1ac0b0530 2023-10-06T02:46:22.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 -- 172.21.15.33:0/209685139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac0b0530 msgr2=0x7fd1ac0b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/209685139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac0b0530 0x7fd1ac0b20d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1940099b0 tx=0x7fd194009250 comp rx=0 tx=0).stop 2023-10-06T02:46:22.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 -- 172.21.15.33:0/209685139 shutdown_connections 2023-10-06T02:46:22.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/209685139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac0b0530 0x7fd1ac0b20d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/209685139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158bf0 0x7fd1ac0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 -- 172.21.15.33:0/209685139 >> 172.21.15.33:0/209685139 conn(0x7fd1ac0cfd70 msgr2=0x7fd1ac0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:22.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 -- 172.21.15.33:0/209685139 shutdown_connections 2023-10-06T02:46:22.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 -- 172.21.15.33:0/209685139 wait complete. 2023-10-06T02:46:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 Processor -- start 2023-10-06T02:46:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.417+0000 7fd1b11d8700 1 -- start start 2023-10-06T02:46:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1ac158bf0 0x7fd1ac155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158fc0 0x7fd1ac15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 0x7fd1ac0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15bce0 con 0x7fd1ac15b890 2023-10-06T02:46:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac15a730 con 0x7fd1ac158bf0 2023-10-06T02:46:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c3830 con 0x7fd1ac158fc0 2023-10-06T02:46:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 0x7fd1ac0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1aad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1ac158bf0 0x7fd1ac155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 0x7fd1ac0ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53518/0 (socket says 172.21.15.33:53518) 2023-10-06T02:46:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 -- 172.21.15.33:0/3178218665 learned_addr learned my addr 172.21.15.33:0/3178218665 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1a259c700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158fc0 0x7fd1ac15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 -- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158fc0 msgr2=0x7fd1ac15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158fc0 0x7fd1ac15b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 -- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1ac158bf0 msgr2=0x7fd1ac155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1ac158bf0 0x7fd1ac155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 -- 172.21.15.33:0/3178218665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940094f0 con 0x7fd1ac15b890 2023-10-06T02:46:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1ab59e700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 0x7fd1ac0ceee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c8e0 tx=0x7fd1a4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4012630 con 0x7fd1ac15b890 2023-10-06T02:46:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a4012790 con 0x7fd1ac15b890 2023-10-06T02:46:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac0c3ad0 con 0x7fd1ac15b890 2023-10-06T02:46:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4012b50 con 0x7fd1ac15b890 2023-10-06T02:46:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.421+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac0c4090 con 0x7fd1ac15b890 2023-10-06T02:46:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.425+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac15bf80 con 0x7fd1ac15b890 2023-10-06T02:46:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.425+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd1a401a3a0 con 0x7fd1ac15b890 2023-10-06T02:46:22.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.425+0000 7fd1a37fe700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd18c060550 0x7fd18c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.425+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd1a407c750 con 0x7fd1ac15b890 2023-10-06T02:46:22.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.425+0000 7fd1aad9d700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd18c060550 0x7fd18c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.425+0000 7fd1aad9d700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd18c060550 0x7fd18c062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009db0 tx=0x7fd19c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.433+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd1a404cdf0 con 0x7fd1ac15b890 2023-10-06T02:46:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.653+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd1ac0b2610 con 0x7fd1ac15b890 2023-10-06T02:46:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.665+0000 7fd1a37fe700 1 -- 172.21.15.33:0/3178218665 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd1ac0b2610 con 0x7fd1ac15b890 2023-10-06T02:46:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd18c060550 msgr2=0x7fd18c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd18c060550 0x7fd18c062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009db0 tx=0x7fd19c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 msgr2=0x7fd1ac0ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 0x7fd1ac0ceee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c8e0 tx=0x7fd1a4001710 comp rx=0 tx=0).stop 2023-10-06T02:46:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 shutdown_connections 2023-10-06T02:46:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd18c060550 0x7fd18c062a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1ac158fc0 0x7fd1ac15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1ac15b890 0x7fd1ac0ceee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 --2- 172.21.15.33:0/3178218665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1ac158bf0 0x7fd1ac155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 >> 172.21.15.33:0/3178218665 conn(0x7fd1ac0cfd70 msgr2=0x7fd1ac155bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 shutdown_connections 2023-10-06T02:46:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.669+0000 7fd1b11d8700 1 -- 172.21.15.33:0/3178218665 wait complete. 2023-10-06T02:46:22.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T02:46:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.793+0000 7f3c92bed700 1 Processor -- start 2023-10-06T02:46:22.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.793+0000 7f3c92bed700 1 -- start start 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c92bed700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 0x7f3c8c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c92bed700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c15d830 con 0x7f3c8c0b0530 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c92bed700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c0b00a0 con 0x7f3c8c158bf0 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c92bed700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c067c10 con 0x7f3c8c158fc0 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3c8c0b0530 0x7f3c8c15ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60674/0 (socket says 172.21.15.33:60674) 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8bfff700 1 -- 172.21.15.33:0/3669684148 learned_addr learned my addr 172.21.15.33:0/3669684148 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:22.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 0x7f3c8c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413142770 0 0) 0x7f3c8c15d830 con 0x7f3c8c0b0530 2023-10-06T02:46:22.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c78003570 con 0x7f3c8c0b0530 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997420086 0 0) 0x7f3c78003570 con 0x7f3c8c0b0530 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 msgr2=0x7f3c8c15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 0x7f3c8c15d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 >> v1:172.21.15.167:6789/0 conn(0x7f3c8c158bf0 legacy=0x7f3c8c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c0cba20 con 0x7f3c8c0b0530 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f3c740017f0 con 0x7f3c8c0b0530 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 >> v1:172.21.15.33:6789/0 conn(0x7f3c8c0b0530 legacy=0x7f3c8c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 0x7f3c78003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c78004010 0x7f3c78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7800bbf0 con 0x7f3c78004d50 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7800bd30 con 0x7f3c78004010 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7800bef0 con 0x7f3c8c158fc0 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c90989700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 0x7f3c78003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8bfff700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c78004010 0x7f3c78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 -- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 msgr2=0x7f3c78003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c158fc0 0x7f3c78003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 -- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c78004010 msgr2=0x7f3c78004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:22.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c78004010 0x7f3c78004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 -- 172.21.15.33:0/3669684148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c78003570 con 0x7f3c78004d50 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c9118a700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c780051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c84006f00 tx=0x7f3c840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1def1548009130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400e710 con 0x7f3c78004d50 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8400e870 con 0x7f3c78004d50 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.797+0000 7f3c8b7fe700 1 -- 172.21.15.33:0/3669684148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8400ec30 con 0x7f3c78004d50 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- 172.21.15.33:0/3669684148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 msgr2=0x7f3c780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3669684148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c780051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c84006f00 tx=0x7f3c840091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c9118a700 1 -- 172.21.15.33:0/3669684148 reap_dead start 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- 172.21.15.33:0/3669684148 shutdown_connections 2023-10-06T02:46:22.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- 172.21.15.33:0/3669684148 >> 172.21.15.33:0/3669684148 conn(0x7f3c8c0cfd70 msgr2=0x7f3c8c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- 172.21.15.33:0/3669684148 shutdown_connections 2023-10-06T02:46:22.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- 172.21.15.33:0/3669684148 wait complete. 2023-10-06T02:46:22.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 Processor -- start 2023-10-06T02:46:22.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- start start 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c8c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c8c0aefb0 0x7f3c8c0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c0c4570 0x7f3c8c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c154190 con 0x7f3c78004d50 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c154470 con 0x7f3c8c0aefb0 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c92bed700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c154750 con 0x7f3c8c0c4570 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c90989700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c8c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c90989700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c8c0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35756/0 (socket says 172.21.15.33:35756) 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c90989700 1 -- 172.21.15.33:0/3585151292 learned_addr learned my addr 172.21.15.33:0/3585151292 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.801+0000 7f3c8bfff700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c8c0aefb0 0x7f3c8c0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c9118a700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c0c4570 0x7f3c8c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c90989700 1 -- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c0c4570 msgr2=0x7f3c8c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c90989700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c0c4570 0x7f3c8c1555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c90989700 1 -- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c8c0aefb0 msgr2=0x7f3c8c0c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c90989700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c8c0aefb0 0x7f3c8c0c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c90989700 1 -- 172.21.15.33:0/3585151292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c154a30 con 0x7f3c78004d50 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c90989700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c8c0aea70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c002950 tx=0x7f3c7c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c013600 con 0x7f3c78004d50 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c013760 con 0x7f3c78004d50 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c153dd0 con 0x7f3c78004d50 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c013b20 con 0x7f3c78004d50 2023-10-06T02:46:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c155c90 con 0x7f3c78004d50 2023-10-06T02:46:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c154bc0 con 0x7f3c78004d50 2023-10-06T02:46:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3c7c00f3e0 con 0x7f3c78004d50 2023-10-06T02:46:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.805+0000 7f3c88ff9700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3c6c060550 0x7f3c6c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.809+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f3c7c07c400 con 0x7f3c78004d50 2023-10-06T02:46:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.809+0000 7f3c8bfff700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3c6c060550 0x7f3c6c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.809+0000 7f3c8bfff700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3c6c060550 0x7f3c6c062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c157310 tx=0x7f3c7400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:22.817+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3c7c04caa0 con 0x7f3c78004d50 2023-10-06T02:46:23.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.033+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f3c8c0710b0 con 0x7f3c78004d50 2023-10-06T02:46:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.041+0000 7f3c88ff9700 1 -- 172.21.15.33:0/3585151292 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3c7c0500c0 con 0x7f3c78004d50 2023-10-06T02:46:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3c6c060550 msgr2=0x7f3c6c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3c6c060550 0x7f3c6c062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c157310 tx=0x7f3c7400c040 comp rx=0 tx=0).stop 2023-10-06T02:46:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 msgr2=0x7f3c8c0aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c8c0aea70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c002950 tx=0x7f3c7c007d30 comp rx=0 tx=0).stop 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 shutdown_connections 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3c6c060550 0x7f3c6c062a00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c8c0c4570 0x7f3c8c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c8c0aefb0 0x7f3c8c0c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 --2- 172.21.15.33:0/3585151292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c78004d50 0x7f3c8c0aea70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 >> 172.21.15.33:0/3585151292 conn(0x7f3c8c0cfd70 msgr2=0x7f3c8c0cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 shutdown_connections 2023-10-06T02:46:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.045+0000 7f3c92bed700 1 -- 172.21.15.33:0/3585151292 wait complete. 2023-10-06T02:46:23.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T02:46:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 Processor -- start 2023-10-06T02:46:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 -- start start 2023-10-06T02:46:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 0x7fe6a0156060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6a006bfc0 0x7fe6a006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a006c960 con 0x7fe6a00c8e50 2023-10-06T02:46:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a00c8b50 con 0x7fe6a006bfc0 2023-10-06T02:46:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe6a5ba1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a0159790 con 0x7fe6a00cc350 2023-10-06T02:46:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe6a00c8e50 0x7fe6a00680c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60680/0 (socket says 172.21.15.33:60680) 2023-10-06T02:46:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 -- 172.21.15.33:0/2204552076 learned_addr learned my addr 172.21.15.33:0/2204552076 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125589155 0 0) 0x7fe690001490 con 0x7fe6a00c8e50 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69f7fe700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 0x7fe6a0156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69ffff700 1 --2- 172.21.15.33:0/2204552076 >> v2:172.21.15.167:3300/0 conn(0x7fe6a006bfc0 0x7fe6a006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe680003570 con 0x7fe6a00c8e50 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1912884239 0 0) 0x7fe680003570 con 0x7fe6a00c8e50 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 msgr2=0x7fe6a0156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 0x7fe6a0156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 >> v2:172.21.15.167:3300/0 conn(0x7fe6a006bfc0 msgr2=0x7fe6a006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 --2- 172.21.15.33:0/2204552076 >> v2:172.21.15.167:3300/0 conn(0x7fe6a006bfc0 0x7fe6a006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00687d0 con 0x7fe6a00c8e50 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7fe690001980 con 0x7fe6a00c8e50 2023-10-06T02:46:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 >> v1:172.21.15.33:6789/0 conn(0x7fe6a00c8e50 legacy=0x7fe6a00680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680003a60 0x7fe680003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe680004810 0x7fe680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 0x7fe6800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a0159790 con 0x7fe680003a60 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68000ba60 con 0x7fe680004810 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68000bba0 con 0x7fe6a00cc350 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69f7fe700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680003a60 0x7fe680003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe680004810 0x7fe680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69ffff700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 0x7fe6800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 -- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 msgr2=0x7fe6800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00cc350 0x7fe6800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 -- 172.21.15.33:0/2204552076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680003a60 msgr2=0x7fe680003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680003a60 0x7fe680003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69effd700 1 -- 172.21.15.33:0/2204552076 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680003570 con 0x7fe680004810 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.169+0000 7fe69f7fe700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680003a60 0x7fe680003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe69effd700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe680004810 0x7fe680004c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe69000ab20 tx=0x7fe6a01565a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccea564aa5e873b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69000c590 con 0x7fe680004810 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69000c6f0 con 0x7fe680004810 2023-10-06T02:46:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe69e7fc700 1 -- 172.21.15.33:0/2204552076 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69000cab0 con 0x7fe680004810 2023-10-06T02:46:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe680004810 msgr2=0x7fe680004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/2204552076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe680004810 0x7fe680004c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe69000ab20 tx=0x7fe6a01565a0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe69ffff700 1 -- 172.21.15.33:0/2204552076 reap_dead start 2023-10-06T02:46:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/2204552076 shutdown_connections 2023-10-06T02:46:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/2204552076 >> 172.21.15.33:0/2204552076 conn(0x7fe6a006e7c0 msgr2=0x7fe6a006ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/2204552076 shutdown_connections 2023-10-06T02:46:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/2204552076 wait complete. 2023-10-06T02:46:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 Processor -- start 2023-10-06T02:46:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.173+0000 7fe6a5ba1700 1 -- start start 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 0x7fe6a0164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00c6b50 0x7fe6a00c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01534d0 0x7fe6a00c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00afff0 con 0x7fe680004810 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b0890 con 0x7fe6a01534d0 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00c4490 con 0x7fe6a00c6b50 2023-10-06T02:46:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 0x7fe6a0164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00c6b50 0x7fe6a00c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 0x7fe6a0164d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35784/0 (socket says 172.21.15.33:35784) 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69f7fe700 1 -- 172.21.15.33:0/3872001501 learned_addr learned my addr 172.21.15.33:0/3872001501 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69ffff700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01534d0 0x7fe6a00c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 -- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01534d0 msgr2=0x7fe6a00c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01534d0 0x7fe6a00c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 -- 172.21.15.33:0/3872001501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 msgr2=0x7fe6a0164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 0x7fe6a0164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 -- 172.21.15.33:0/3872001501 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00b0240 con 0x7fe6a00c6b50 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69f7fe700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 0x7fe6a0164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe69effd700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00c6b50 0x7fe6a00c3b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe690015000 tx=0x7fe690016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69000cea0 con 0x7fe6a00c6b50 2023-10-06T02:46:23.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69001f440 con 0x7fe6a00c6b50 2023-10-06T02:46:23.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a00b0520 con 0x7fe6a00c6b50 2023-10-06T02:46:23.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69001f5d0 con 0x7fe6a00c6b50 2023-10-06T02:46:23.185 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.177+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a00c4080 con 0x7fe6a00c6b50 2023-10-06T02:46:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.181+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a00b0a20 con 0x7fe6a00c6b50 2023-10-06T02:46:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.181+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe690025020 con 0x7fe6a00c6b50 2023-10-06T02:46:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.181+0000 7fe6977fe700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe684060580 0x7fe684062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.181+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fe69007ee70 con 0x7fe6a00c6b50 2023-10-06T02:46:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.181+0000 7fe69f7fe700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe684060580 0x7fe684062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.181+0000 7fe69f7fe700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe684060580 0x7fe684062a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe688009720 tx=0x7fe6880091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.193+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe69004f470 con 0x7fe6a00c6b50 2023-10-06T02:46:23.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.413+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe6a006c960 con 0x7fe6a00c6b50 2023-10-06T02:46:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.421+0000 7fe6977fe700 1 -- 172.21.15.33:0/3872001501 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe6a006c960 con 0x7fe6a00c6b50 2023-10-06T02:46:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe684060580 msgr2=0x7fe684062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe684060580 0x7fe684062a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe688009720 tx=0x7fe6880091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00c6b50 msgr2=0x7fe6a00c3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00c6b50 0x7fe6a00c3b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe690015000 tx=0x7fe690016000 comp rx=0 tx=0).stop 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 shutdown_connections 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe684060580 0x7fe684062a30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a00c6b50 0x7fe6a00c3b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a01534d0 0x7fe6a00c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 --2- 172.21.15.33:0/3872001501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe680004810 0x7fe6a0164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 >> 172.21.15.33:0/3872001501 conn(0x7fe6a006e7c0 msgr2=0x7fe6a00c2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 shutdown_connections 2023-10-06T02:46:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.425+0000 7fe6a5ba1700 1 -- 172.21.15.33:0/3872001501 wait complete. 2023-10-06T02:46:23.444 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T02:46:23.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 Processor -- start 2023-10-06T02:46:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 -- start start 2023-10-06T02:46:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2fc0c9cb0 0x7fd2fc0ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc06b420 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc0ca670 con 0x7fd2fc0c9cb0 2023-10-06T02:46:23.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.549+0000 7fd301d5a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc069cf0 con 0x7fd2fc06aa70 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301559700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd2fc0c9cb0 0x7fd2fc0ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc06aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41318/0 (socket says 172.21.15.33:41318) 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 -- 172.21.15.33:0/3199691873 learned_addr learned my addr 172.21.15.33:0/3199691873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 -- 172.21.15.33:0/3199691873 >> v2:172.21.15.167:3300/0 conn(0x7fd2fc0c9cb0 msgr2=0x7fd2fc0ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 --2- 172.21.15.33:0/3199691873 >> v2:172.21.15.167:3300/0 conn(0x7fd2fc0c9cb0 0x7fd2fc0ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 -- 172.21.15.33:0/3199691873 >> v1:172.21.15.167:6789/0 conn(0x7fd2fc06b5d0 legacy=0x7fd2fc0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 -- 172.21.15.33:0/3199691873 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc069f90 con 0x7fd2fc06aa70 2023-10-06T02:46:23.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fbfff700 1 --2- 172.21.15.33:0/3199691873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc06aee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40099b0 tx=0x7fd2e4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e065bb7eec7123b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fb7fe700 1 -- 172.21.15.33:0/3199691873 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4009ea0 con 0x7fd2fc06aa70 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fb7fe700 1 -- 172.21.15.33:0/3199691873 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e400b860 con 0x7fd2fc06aa70 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd2fb7fe700 1 -- 172.21.15.33:0/3199691873 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e400bc00 con 0x7fd2fc06aa70 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 -- 172.21.15.33:0/3199691873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 msgr2=0x7fd2fc06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 --2- 172.21.15.33:0/3199691873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc06aee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40099b0 tx=0x7fd2e4009250 comp rx=0 tx=0).stop 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 -- 172.21.15.33:0/3199691873 shutdown_connections 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 --2- 172.21.15.33:0/3199691873 >> v2:172.21.15.167:3300/0 conn(0x7fd2fc0c9cb0 0x7fd2fc0ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 --2- 172.21.15.33:0/3199691873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc06aee0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 -- 172.21.15.33:0/3199691873 >> 172.21.15.33:0/3199691873 conn(0x7fd2fc0c8240 msgr2=0x7fd2fc0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 -- 172.21.15.33:0/3199691873 shutdown_connections 2023-10-06T02:46:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 -- 172.21.15.33:0/3199691873 wait complete. 2023-10-06T02:46:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.553+0000 7fd301d5a700 1 Processor -- start 2023-10-06T02:46:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 -- start start 2023-10-06T02:46:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc1485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 0x7fd2fc0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2fc0c9cb0 0x7fd2fc16afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc148af0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc14d640 con 0x7fd2fc0c9cb0 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc14d920 con 0x7fd2fc06aa70 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 0x7fd2fc0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 0x7fd2fc0c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35800/0 (socket says 172.21.15.33:35800) 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 -- 172.21.15.33:0/620439185 learned_addr learned my addr 172.21.15.33:0/620439185 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301559700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2fc0c9cb0 0x7fd2fc16afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd300d58700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc1485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 -- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 msgr2=0x7fd2fc1485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc1485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 -- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2fc0c9cb0 msgr2=0x7fd2fc16afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2fc0c9cb0 0x7fd2fc16afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 -- 172.21.15.33:0/620439185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40094f0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2fbfff700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 0x7fd2fc0c4410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc069e30 tx=0x7fd2e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4000d10 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e400bbe0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc14dc00 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4011650 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.557+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc0c4b70 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.561+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc0ae6d0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.561+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd2e4011a80 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.561+0000 7fd2f8ff9700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2dc060580 0x7fd2dc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.561+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd2e407da70 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.561+0000 7fd300d58700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2dc060580 0x7fd2dc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.561+0000 7fd300d58700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2dc060580 0x7fd2dc062a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec003aa0 tx=0x7fd2ec000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.565+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2fc0ae6d0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.789+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd2fc0ae6d0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd2f8ff9700 1 -- 172.21.15.33:0/620439185 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd2fc0ae6d0 con 0x7fd2fc06b5d0 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2dc060580 msgr2=0x7fd2dc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2dc060580 0x7fd2dc062a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec003aa0 tx=0x7fd2ec000bc0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 msgr2=0x7fd2fc0c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 0x7fd2fc0c4410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc069e30 tx=0x7fd2e4017000 comp rx=0 tx=0).stop 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 shutdown_connections 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2fc0c9cb0 0x7fd2fc16afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2fc06b5d0 0x7fd2fc0c4410 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2dc060580 0x7fd2dc062a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.797+0000 7fd301d5a700 1 --2- 172.21.15.33:0/620439185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2fc06aa70 0x7fd2fc1485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.801+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 >> 172.21.15.33:0/620439185 conn(0x7fd2fc0c8240 msgr2=0x7fd2fc0c7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.801+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 shutdown_connections 2023-10-06T02:46:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.801+0000 7fd301d5a700 1 -- 172.21.15.33:0/620439185 wait complete. 2023-10-06T02:46:23.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T02:46:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.921+0000 7f8839034700 1 Processor -- start 2023-10-06T02:46:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.921+0000 7f8839034700 1 -- start start 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8834160fd0 0x7f88340c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8834067050 con 0x7f8834161820 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8834067330 con 0x7f88341613a0 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8834161c80 con 0x7f8834160fd0 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8834160fd0 0x7f88340c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8834160fd0 0x7f88340c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35806/0 (socket says 172.21.15.33:35806) 2023-10-06T02:46:23.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 -- 172.21.15.33:0/3312950359 learned_addr learned my addr 172.21.15.33:0/3312950359 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 -- 172.21.15.33:0/3312950359 >> v1:172.21.15.167:6789/0 conn(0x7f88341613a0 legacy=0x7f88340680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 -- 172.21.15.33:0/3312950359 >> v1:172.21.15.33:6789/0 conn(0x7f8834161820 legacy=0x7f88340c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8831d9b700 1 -- 172.21.15.33:0/3312950359 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683444509 0 0) 0x7f8834067050 con 0x7f8834161820 2023-10-06T02:46:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 -- 172.21.15.33:0/3312950359 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8834067610 con 0x7f8834160fd0 2023-10-06T02:46:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8832d9d700 1 --2- 172.21.15.33:0/3312950359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834160fd0 0x7f88340c9550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8824009f20 tx=0x7f8824009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab818b0b5d9bd0f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8831d9b700 1 -- 172.21.15.33:0/3312950359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400e070 con 0x7f8834160fd0 2023-10-06T02:46:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8831d9b700 1 -- 172.21.15.33:0/3312950359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882400ba80 con 0x7f8834160fd0 2023-10-06T02:46:23.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8831d9b700 1 -- 172.21.15.33:0/3312950359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240092e0 con 0x7f8834160fd0 2023-10-06T02:46:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- 172.21.15.33:0/3312950359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834160fd0 msgr2=0x7f88340c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 --2- 172.21.15.33:0/3312950359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834160fd0 0x7f88340c9550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8824009f20 tx=0x7f8824009840 comp rx=0 tx=0).stop 2023-10-06T02:46:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- 172.21.15.33:0/3312950359 shutdown_connections 2023-10-06T02:46:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 --2- 172.21.15.33:0/3312950359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834160fd0 0x7f88340c9550 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- 172.21.15.33:0/3312950359 >> 172.21.15.33:0/3312950359 conn(0x7f88340c83f0 msgr2=0x7f88340c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- 172.21.15.33:0/3312950359 shutdown_connections 2023-10-06T02:46:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.925+0000 7f8839034700 1 -- 172.21.15.33:0/3312950359 wait complete. 2023-10-06T02:46:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 Processor -- start 2023-10-06T02:46:23.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 -- start start 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8834160fd0 0x7f88340b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88341613a0 0x7f8834150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 0x7f88340c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340b1090 con 0x7f8834161820 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340c31b0 con 0x7f8834160fd0 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8839034700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8834151290 con 0x7f88341613a0 2023-10-06T02:46:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f883359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 0x7f88340c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f883359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 0x7f88340c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35818/0 (socket says 172.21.15.33:35818) 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f883359e700 1 -- 172.21.15.33:0/821519389 learned_addr learned my addr 172.21.15.33:0/821519389 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f8832d9d700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8834160fd0 0x7f88340b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f883259c700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88341613a0 0x7f8834150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f883359e700 1 -- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88341613a0 msgr2=0x7f8834150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.929+0000 7f883359e700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88341613a0 0x7f8834150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f883359e700 1 -- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8834160fd0 msgr2=0x7f88340b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f883359e700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8834160fd0 0x7f88340b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f883359e700 1 -- 172.21.15.33:0/821519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824009ae0 con 0x7f8834161820 2023-10-06T02:46:23.938 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f883359e700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 0x7f88340c5260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c4f0 tx=0x7f882c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00ce60 con 0x7f8834161820 2023-10-06T02:46:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c011590 con 0x7f8834161820 2023-10-06T02:46:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88340b0db0 con 0x7f8834161820 2023-10-06T02:46:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c011950 con 0x7f8834161820 2023-10-06T02:46:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88340c2e40 con 0x7f8834161820 2023-10-06T02:46:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88340710b0 con 0x7f8834161820 2023-10-06T02:46:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f882c011cc0 con 0x7f8834161820 2023-10-06T02:46:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f882affd700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8814060550 0x7f8814062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f882c07ba10 con 0x7f8834161820 2023-10-06T02:46:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.933+0000 7f8832d9d700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8814060550 0x7f8814062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.937+0000 7f8832d9d700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8814060550 0x7f8814062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8824009880 tx=0x7f8824018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:23.941+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f882c04c0b0 con 0x7f8834161820 2023-10-06T02:46:24.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.161+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f88340c0920 con 0x7f8834161820 2023-10-06T02:46:24.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.165+0000 7f882affd700 1 -- 172.21.15.33:0/821519389 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f882c04f6d0 con 0x7f8834161820 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8814060550 msgr2=0x7f8814062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8814060550 0x7f8814062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8824009880 tx=0x7f8824018000 comp rx=0 tx=0).stop 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 msgr2=0x7f88340c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 0x7f88340c5260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c4f0 tx=0x7f882c007f40 comp rx=0 tx=0).stop 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 shutdown_connections 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8814060550 0x7f8814062a00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88341613a0 0x7f8834150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8834161820 0x7f88340c5260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 --2- 172.21.15.33:0/821519389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8834160fd0 0x7f88340b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 >> 172.21.15.33:0/821519389 conn(0x7f88340c83f0 msgr2=0x7f8834069150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 shutdown_connections 2023-10-06T02:46:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.169+0000 7f8839034700 1 -- 172.21.15.33:0/821519389 wait complete. 2023-10-06T02:46:24.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:24.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc91eb700 1 Processor -- start 2023-10-06T02:46:24.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc91eb700 1 -- start start 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc91eb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1bc406be70 0x7f1bc4163de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc91eb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc40c8de0 con 0x7f1bc41593d0 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc91eb700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc40c9c30 con 0x7f1bc4159000 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc91eb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc4067fc0 con 0x7f1bc406be70 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1bc406be70 0x7f1bc4163de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc259c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1bc41593d0 0x7f1bc406b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60694/0 (socket says 172.21.15.33:60694) 2023-10-06T02:46:24.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc259c700 1 -- 172.21.15.33:0/2726908110 learned_addr learned my addr 172.21.15.33:0/2726908110 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:24.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc359e700 1 -- 172.21.15.33:0/2726908110 >> v1:172.21.15.167:6789/0 conn(0x7f1bc4159000 legacy=0x7f1bc40c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc359e700 1 -- 172.21.15.33:0/2726908110 >> v1:172.21.15.33:6789/0 conn(0x7f1bc41593d0 legacy=0x7f1bc406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc1d9b700 1 -- 172.21.15.33:0/2726908110 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840625654 0 0) 0x7f1bc40c9c30 con 0x7f1bc4159000 2023-10-06T02:46:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc359e700 1 -- 172.21.15.33:0/2726908110 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc40c9020 con 0x7f1bc406be70 2023-10-06T02:46:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc359e700 1 --2- 172.21.15.33:0/2726908110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc406be70 0x7f1bc4163de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40098e0 tx=0x7f1bb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab99656d19467a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc1d9b700 1 -- 172.21.15.33:0/2726908110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4009e80 con 0x7f1bc406be70 2023-10-06T02:46:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc1d9b700 1 -- 172.21.15.33:0/2726908110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb400b760 con 0x7f1bc406be70 2023-10-06T02:46:24.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.289+0000 7f1bc1d9b700 1 -- 172.21.15.33:0/2726908110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb400bae0 con 0x7f1bc406be70 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 -- 172.21.15.33:0/2726908110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc406be70 msgr2=0x7f1bc4163de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/2726908110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc406be70 0x7f1bc4163de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40098e0 tx=0x7f1bb40091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 -- 172.21.15.33:0/2726908110 shutdown_connections 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/2726908110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc406be70 0x7f1bc4163de0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 -- 172.21.15.33:0/2726908110 >> 172.21.15.33:0/2726908110 conn(0x7f1bc40af920 msgr2=0x7f1bc40afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 -- 172.21.15.33:0/2726908110 shutdown_connections 2023-10-06T02:46:24.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 -- 172.21.15.33:0/2726908110 wait complete. 2023-10-06T02:46:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 Processor -- start 2023-10-06T02:46:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.293+0000 7f1bc91eb700 1 -- start start 2023-10-06T02:46:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bc406be70 0x7f1bc414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 0x7f1bc4151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bc41593d0 0x7f1bc40c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc4153910 con 0x7f1bc4159000 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc4153bb0 con 0x7f1bc406be70 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc4153e90 con 0x7f1bc41593d0 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 0x7f1bc4151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 0x7f1bc4151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35822/0 (socket says 172.21.15.33:35822) 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 -- 172.21.15.33:0/3787220257 learned_addr learned my addr 172.21.15.33:0/3787220257 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc2d9d700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bc406be70 0x7f1bc414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc359e700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bc41593d0 0x7f1bc40c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 -- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bc41593d0 msgr2=0x7f1bc40c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bc41593d0 0x7f1bc40c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 -- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bc406be70 msgr2=0x7f1bc414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bc406be70 0x7f1bc414f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 -- 172.21.15.33:0/3787220257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4009420 con 0x7f1bc4159000 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc259c700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 0x7f1bc4151dc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c960 tx=0x7f1bb8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8005b30 con 0x7f1bc4159000 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8005c90 con 0x7f1bc4159000 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc4154170 con 0x7f1bc4159000 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8007340 con 0x7f1bc4159000 2023-10-06T02:46:24.303 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc4154730 con 0x7f1bc4159000 2023-10-06T02:46:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.297+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc4154950 con 0x7f1bc4159000 2023-10-06T02:46:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.301+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1bb801b3b0 con 0x7f1bc4159000 2023-10-06T02:46:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.301+0000 7f1baaffd700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1bb0060550 0x7f1bb0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.301+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f1bb807bbf0 con 0x7f1bc4159000 2023-10-06T02:46:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.301+0000 7f1bc2d9d700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1bb0060550 0x7f1bb0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.301+0000 7f1bc2d9d700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1bb0060550 0x7f1bb0062a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c5f0 tx=0x7f1bbc001b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.309+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1bb804c270 con 0x7f1bc4159000 2023-10-06T02:46:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.473+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bc40c8de0 con 0x7f1bc4159000 2023-10-06T02:46:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.473+0000 7f1baaffd700 1 -- 172.21.15.33:0/3787220257 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13855 (secure 0 0 0) 0x7f1bc40c8de0 con 0x7f1bc4159000 2023-10-06T02:46:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:24.479 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:21.502401+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1bb0060550 msgr2=0x7f1bb0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1bb0060550 0x7f1bb0062a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c5f0 tx=0x7f1bbc001b10 comp rx=0 tx=0).stop 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 msgr2=0x7f1bc4151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 0x7f1bc4151dc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c960 tx=0x7f1bb8007ef0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 shutdown_connections 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1bb0060550 0x7f1bb0062a00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1bc41593d0 0x7f1bc40c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bc4159000 0x7f1bc4151dc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 --2- 172.21.15.33:0/3787220257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1bc406be70 0x7f1bc414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 >> 172.21.15.33:0/3787220257 conn(0x7f1bc40af920 msgr2=0x7f1bc4150d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 shutdown_connections 2023-10-06T02:46:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.477+0000 7f1bc91eb700 1 -- 172.21.15.33:0/3787220257 wait complete. 2023-10-06T02:46:24.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T02:46:24.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:24.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.601+0000 7fd2fdfc3700 1 Processor -- start 2023-10-06T02:46:24.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.601+0000 7fd2fdfc3700 1 -- start start 2023-10-06T02:46:24.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2f80c7e50 0x7fd2f8066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f8155c80 0x7fd2f80674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f80c5e30 con 0x7fd2f81558b0 2023-10-06T02:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f80c69b0 con 0x7fd2f80c7e50 2023-10-06T02:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80c6080 con 0x7fd2f8155c80 2023-10-06T02:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2f80c7e50 0x7fd2f8066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd2f80c7e50 0x7fd2f8066fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35838/0 (socket says 172.21.15.33:35838) 2023-10-06T02:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 -- 172.21.15.33:0/3211195408 learned_addr learned my addr 172.21.15.33:0/3211195408 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fd7c2700 1 --2- 172.21.15.33:0/3211195408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f8155c80 0x7fd2f80674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 -- 172.21.15.33:0/3211195408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f8155c80 msgr2=0x7fd2f80674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 --2- 172.21.15.33:0/3211195408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f8155c80 0x7fd2f80674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 -- 172.21.15.33:0/3211195408 >> v1:172.21.15.167:6789/0 conn(0x7fd2f81558b0 legacy=0x7fd2f80ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 -- 172.21.15.33:0/3211195408 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f80c6360 con 0x7fd2f80c7e50 2023-10-06T02:46:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f7fff700 1 --2- 172.21.15.33:0/3211195408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f80c7e50 0x7fd2f8066fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80099b0 tx=0x7fd2e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238fc68ada28eb4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f77fe700 1 -- 172.21.15.33:0/3211195408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800b860 con 0x7fd2f80c7e50 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f77fe700 1 -- 172.21.15.33:0/3211195408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e800b9c0 con 0x7fd2f80c7e50 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2f77fe700 1 -- 172.21.15.33:0/3211195408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800bd60 con 0x7fd2f80c7e50 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/3211195408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f80c7e50 msgr2=0x7fd2f8066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/3211195408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f80c7e50 0x7fd2f8066fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80099b0 tx=0x7fd2e8009250 comp rx=0 tx=0).stop 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/3211195408 shutdown_connections 2023-10-06T02:46:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/3211195408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f8155c80 0x7fd2f80674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/3211195408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f80c7e50 0x7fd2f8066fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/3211195408 >> 172.21.15.33:0/3211195408 conn(0x7fd2f80cc2c0 msgr2=0x7fd2f80c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/3211195408 shutdown_connections 2023-10-06T02:46:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/3211195408 wait complete. 2023-10-06T02:46:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 Processor -- start 2023-10-06T02:46:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.605+0000 7fd2fdfc3700 1 -- start start 2023-10-06T02:46:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2f81558b0 0x7fd2f8156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 0x7fd2f80afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f80b0150 0x7fd2f80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f815c250 con 0x7fd2f8155c80 2023-10-06T02:46:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80c0550 con 0x7fd2f81558b0 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80bff90 con 0x7fd2f80b0150 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 0x7fd2f80afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 0x7fd2f80afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35844/0 (socket says 172.21.15.33:35844) 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 -- 172.21.15.33:0/1344829885 learned_addr learned my addr 172.21.15.33:0/1344829885 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fd7c2700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f80b0150 0x7fd2f80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fcfc1700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2f81558b0 0x7fd2f8156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 -- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f80b0150 msgr2=0x7fd2f80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f80b0150 0x7fd2f80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 -- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2f81558b0 msgr2=0x7fd2f8156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2f81558b0 0x7fd2f8156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 -- 172.21.15.33:0/1344829885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80094f0 con 0x7fd2f8155c80 2023-10-06T02:46:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f7fff700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 0x7fd2f80afc10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd2f806c9a0 tx=0x7fd2e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8024440 con 0x7fd2f8155c80 2023-10-06T02:46:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e80245c0 con 0x7fd2f8155c80 2023-10-06T02:46:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f80c0270 con 0x7fd2f8155c80 2023-10-06T02:46:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8024980 con 0x7fd2f8155c80 2023-10-06T02:46:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f8156fa0 con 0x7fd2f8155c80 2023-10-06T02:46:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.609+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f815c400 con 0x7fd2f8155c80 2023-10-06T02:46:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.613+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd2e802d430 con 0x7fd2f8155c80 2023-10-06T02:46:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.613+0000 7fd2f4ff9700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2d8060580 0x7fd2d8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.613+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fd2e807e4c0 con 0x7fd2f8155c80 2023-10-06T02:46:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.613+0000 7fd2fcfc1700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2d8060580 0x7fd2d8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.613+0000 7fd2fcfc1700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2d8060580 0x7fd2d8062a30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0003860 tx=0x7fd2e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.617+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2f815c400 con 0x7fd2f8155c80 2023-10-06T02:46:24.785 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.781+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2f815c400 con 0x7fd2f8155c80 2023-10-06T02:46:24.786 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.781+0000 7fd2f4ff9700 1 -- 172.21.15.33:0/1344829885 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2f815c400 con 0x7fd2f8155c80 2023-10-06T02:46:24.786 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2d8060580 msgr2=0x7fd2d8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2d8060580 0x7fd2d8062a30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0003860 tx=0x7fd2e0009040 comp rx=0 tx=0).stop 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 msgr2=0x7fd2f80afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 0x7fd2f80afc10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd2f806c9a0 tx=0x7fd2e8017000 comp rx=0 tx=0).stop 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 shutdown_connections 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd2d8060580 0x7fd2d8062a30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd2f80b0150 0x7fd2f80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2f8155c80 0x7fd2f80afc10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 --2- 172.21.15.33:0/1344829885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2f81558b0 0x7fd2f8156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 >> 172.21.15.33:0/1344829885 conn(0x7fd2f80cc2c0 msgr2=0x7fd2f80693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:24.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 shutdown_connections 2023-10-06T02:46:24.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.785+0000 7fd2fdfc3700 1 -- 172.21.15.33:0/1344829885 wait complete. 2023-10-06T02:46:24.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab2d619700 1 Processor -- start 2023-10-06T02:46:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab2d619700 1 -- start start 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab2d619700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 0x7fab280b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab2d619700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab280aa2f0 con 0x7fab28155a20 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab2d619700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab280a9720 con 0x7fab280b2690 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab2d619700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab280a99c0 con 0x7fab28155df0 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab26ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab28155a20 0x7fab28068ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60704/0 (socket says 172.21.15.33:60704) 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.913+0000 7fab26ffd700 1 -- 172.21.15.33:0/426941727 learned_addr learned my addr 172.21.15.33:0/426941727 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 0x7fab280b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6906913 0 0) 0x7fab280aa2f0 con 0x7fab28155a20 2023-10-06T02:46:24.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab10003570 con 0x7fab28155a20 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335608416 0 0) 0x7fab280a9720 con 0x7fab280b2690 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab280aa2f0 con 0x7fab280b2690 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732168148 0 0) 0x7fab10003570 con 0x7fab28155a20 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 msgr2=0x7fab280b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 0x7fab280b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 >> v1:172.21.15.167:6789/0 conn(0x7fab280b2690 legacy=0x7fab280aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280a9ca0 con 0x7fab28155a20 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7fab18001770 con 0x7fab28155a20 2023-10-06T02:46:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 >> v1:172.21.15.33:6789/0 conn(0x7fab28155a20 legacy=0x7fab28068ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 0x7fab10003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab100041d0 0x7fab10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab10004d50 0x7fab100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab280a99c0 con 0x7fab10004d50 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1000bbf0 con 0x7fab100041d0 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1000bdb0 con 0x7fab28155df0 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab10004d50 0x7fab100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab26ffd700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 0x7fab10003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab267fc700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab100041d0 0x7fab10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 -- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 msgr2=0x7fab10003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28155df0 0x7fab10003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 -- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab100041d0 msgr2=0x7fab10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:24.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab100041d0 0x7fab10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 -- 172.21.15.33:0/426941727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab10003570 con 0x7fab10004d50 2023-10-06T02:46:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab267fc700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab100041d0 0x7fab10004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab10004d50 0x7fab100051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab20009c00 tx=0x7fab200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd43339898d7048e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000e710 con 0x7fab10004d50 2023-10-06T02:46:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2000e870 con 0x7fab10004d50 2023-10-06T02:46:24.925 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab25ffb700 1 -- 172.21.15.33:0/426941727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000ec30 con 0x7fab10004d50 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab2d619700 1 -- 172.21.15.33:0/426941727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab10004d50 msgr2=0x7fab100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab2d619700 1 --2- 172.21.15.33:0/426941727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab10004d50 0x7fab100051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab20009c00 tx=0x7fab200091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab277fe700 1 -- 172.21.15.33:0/426941727 reap_dead start 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab2d619700 1 -- 172.21.15.33:0/426941727 shutdown_connections 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab2d619700 1 -- 172.21.15.33:0/426941727 >> 172.21.15.33:0/426941727 conn(0x7fab280af9c0 msgr2=0x7fab280afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.917+0000 7fab2d619700 1 -- 172.21.15.33:0/426941727 shutdown_connections 2023-10-06T02:46:24.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.921+0000 7fab2d619700 1 -- 172.21.15.33:0/426941727 wait complete. 2023-10-06T02:46:24.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.921+0000 7fab2d619700 1 Processor -- start 2023-10-06T02:46:24.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.921+0000 7fab2d619700 1 -- start start 2023-10-06T02:46:24.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28143460 0x7fab28097000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab28143830 0x7fab28097fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 0x7fab28099500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28098ad0 con 0x7fab28165e20 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab280987f0 con 0x7fab28143830 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab28152b30 con 0x7fab28143460 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 0x7fab28099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 0x7fab28099500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35866/0 (socket says 172.21.15.33:35866) 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 -- 172.21.15.33:0/2932323919 learned_addr learned my addr 172.21.15.33:0/2932323919 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab26ffd700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28143460 0x7fab28097000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab267fc700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab28143830 0x7fab28097fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 -- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28143460 msgr2=0x7fab28097000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28143460 0x7fab28097000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 -- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab28143830 msgr2=0x7fab28097fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab28143830 0x7fab28097fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 -- 172.21.15.33:0/2932323919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28152e10 con 0x7fab28165e20 2023-10-06T02:46:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab277fe700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 0x7fab28099500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab20001eb0 tx=0x7fab2001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20022070 con 0x7fab28165e20 2023-10-06T02:46:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2000ebf0 con 0x7fab28165e20 2023-10-06T02:46:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab281530f0 con 0x7fab28165e20 2023-10-06T02:46:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000e710 con 0x7fab28165e20 2023-10-06T02:46:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.925+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab28099a40 con 0x7fab28165e20 2023-10-06T02:46:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.929+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab28098ad0 con 0x7fab28165e20 2023-10-06T02:46:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.929+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fab2001c8f0 con 0x7fab28165e20 2023-10-06T02:46:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.929+0000 7fab0effd700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab14060550 0x7fab14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.929+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7fab2007d6b0 con 0x7fab28165e20 2023-10-06T02:46:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.929+0000 7fab26ffd700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab14060550 0x7fab14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.929+0000 7fab26ffd700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab14060550 0x7fab14062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab180011d0 tx=0x7fab18006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:24.937+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fab2004ddd0 con 0x7fab28165e20 2023-10-06T02:46:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fab280710b0 con 0x7fab28165e20 2023-10-06T02:46:25.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab0effd700 1 -- 172.21.15.33:0/2932323919 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fab200513f0 con 0x7fab28165e20 2023-10-06T02:46:25.101 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:25.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab14060550 msgr2=0x7fab14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab14060550 0x7fab14062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab180011d0 tx=0x7fab18006040 comp rx=0 tx=0).stop 2023-10-06T02:46:25.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 msgr2=0x7fab28099500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 0x7fab28099500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab20001eb0 tx=0x7fab2001a000 comp rx=0 tx=0).stop 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 shutdown_connections 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab28165e20 0x7fab28099500 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab28143830 0x7fab28097fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fab14060550 0x7fab14062a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 --2- 172.21.15.33:0/2932323919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab28143460 0x7fab28097000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 >> 172.21.15.33:0/2932323919 conn(0x7fab280af9c0 msgr2=0x7fab2806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 shutdown_connections 2023-10-06T02:46:25.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.097+0000 7fab2d619700 1 -- 172.21.15.33:0/2932323919 wait complete. 2023-10-06T02:46:25.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T02:46:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.217+0000 7f8102f67700 1 Processor -- start 2023-10-06T02:46:25.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.217+0000 7f8102f67700 1 -- start start 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.217+0000 7f8102f67700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc163c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.217+0000 7f8102f67700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0af6c0 con 0x7f80fc15d600 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.217+0000 7f8102f67700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0af910 con 0x7f80fc158bc0 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.217+0000 7f8102f67700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0afbf0 con 0x7f80fc159fa0 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8101764700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f80fc15d600 0x7f80fc06bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60714/0 (socket says 172.21.15.33:60714) 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8101764700 1 -- 172.21.15.33:0/810091954 learned_addr learned my addr 172.21.15.33:0/810091954 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102766700 1 --2- 172.21.15.33:0/810091954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8100f63700 1 -- 172.21.15.33:0/810091954 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176965043 0 0) 0x7f80fc0af6c0 con 0x7f80fc15d600 2023-10-06T02:46:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8100f63700 1 -- 172.21.15.33:0/810091954 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80e8003570 con 0x7f80fc15d600 2023-10-06T02:46:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102766700 1 -- 172.21.15.33:0/810091954 >> v1:172.21.15.167:6789/0 conn(0x7f80fc158bc0 legacy=0x7f80fc0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102766700 1 -- 172.21.15.33:0/810091954 >> v1:172.21.15.33:6789/0 conn(0x7f80fc15d600 legacy=0x7f80fc06bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102766700 1 -- 172.21.15.33:0/810091954 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0afed0 con 0x7f80fc159fa0 2023-10-06T02:46:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102766700 1 --2- 172.21.15.33:0/810091954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc163c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009e90 tx=0x7f80f4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c798a6a2cf7439e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8100f63700 1 -- 172.21.15.33:0/810091954 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f400e070 con 0x7f80fc159fa0 2023-10-06T02:46:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8100f63700 1 -- 172.21.15.33:0/810091954 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f400bc30 con 0x7f80fc159fa0 2023-10-06T02:46:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8100f63700 1 -- 172.21.15.33:0/810091954 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4012430 con 0x7f80fc159fa0 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 -- 172.21.15.33:0/810091954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 msgr2=0x7f80fc163c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 --2- 172.21.15.33:0/810091954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc163c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009e90 tx=0x7f80f4009780 comp rx=0 tx=0).stop 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 -- 172.21.15.33:0/810091954 shutdown_connections 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 --2- 172.21.15.33:0/810091954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc163c50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 -- 172.21.15.33:0/810091954 >> 172.21.15.33:0/810091954 conn(0x7f80fc06e6c0 msgr2=0x7f80fc06ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 -- 172.21.15.33:0/810091954 shutdown_connections 2023-10-06T02:46:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.221+0000 7f8102f67700 1 -- 172.21.15.33:0/810091954 wait complete. 2023-10-06T02:46:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 Processor -- start 2023-10-06T02:46:25.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 -- start start 2023-10-06T02:46:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc158bc0 0x7f80fc0acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc1461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 0x7f80fc0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0ac190 con 0x7f80fc146720 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0ad0e0 con 0x7f80fc158bc0 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0ae3c0 con 0x7f80fc159fa0 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102766700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 0x7f80fc0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc1461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102766700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 0x7f80fc0ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35876/0 (socket says 172.21.15.33:35876) 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc1461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41394/0 (socket says 172.21.15.33:41394) 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102766700 1 -- 172.21.15.33:0/3549866516 learned_addr learned my addr 172.21.15.33:0/3549866516 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:25.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101f65700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc158bc0 0x7f80fc0acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 -- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc158bc0 msgr2=0x7f80fc0acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc158bc0 0x7f80fc0acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 -- 172.21.15.33:0/3549866516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 msgr2=0x7f80fc0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 0x7f80fc0ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 -- 172.21.15.33:0/3549866516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f4009a20 con 0x7f80fc159fa0 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102766700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 0x7f80fc0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8101764700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc1461e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c560 tx=0x7f80f8007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8005b30 con 0x7f80fc159fa0 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f8005cb0 con 0x7f80fc159fa0 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc0ae6a0 con 0x7f80fc159fa0 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8007320 con 0x7f80fc159fa0 2023-10-06T02:46:25.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.225+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc0ac680 con 0x7f80fc159fa0 2023-10-06T02:46:25.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.229+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc0ac8a0 con 0x7f80fc159fa0 2023-10-06T02:46:25.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.229+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f80f8033a20 con 0x7f80fc159fa0 2023-10-06T02:46:25.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.229+0000 7f80f1ffb700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f80e00605d0 0x7f80e0062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.229+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f80f807baf0 con 0x7f80fc159fa0 2023-10-06T02:46:25.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.229+0000 7f8101f65700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f80e00605d0 0x7f80e0062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.229+0000 7f8101f65700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f80e00605d0 0x7f80e0062a80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f80ec007fd0 tx=0x7f80ec007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:25.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.237+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f80f804c170 con 0x7f80fc159fa0 2023-10-06T02:46:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.405+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f80fc071040 con 0x7f80fc159fa0 2023-10-06T02:46:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.405+0000 7f80f1ffb700 1 -- 172.21.15.33:0/3549866516 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+126 (secure 0 0 0) 0x7f80f8017090 con 0x7f80fc159fa0 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:25.412 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 2, 2023-10-06T02:46:25.412 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": "rbd" 2023-10-06T02:46:25.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:25.412 INFO:teuthology.orchestra.run.smithi033.stdout:] 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f80e00605d0 msgr2=0x7f80e0062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f80e00605d0 0x7f80e0062a80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f80ec007fd0 tx=0x7f80ec007eb0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 msgr2=0x7f80fc1461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc1461e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c560 tx=0x7f80f8007d50 comp rx=0 tx=0).stop 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 shutdown_connections 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f80e00605d0 0x7f80e0062a80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc159fa0 0x7f80fc1461e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80fc146720 0x7f80fc0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 --2- 172.21.15.33:0/3549866516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc158bc0 0x7f80fc0acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 >> 172.21.15.33:0/3549866516 conn(0x7f80fc06e6c0 msgr2=0x7f80fc1520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 shutdown_connections 2023-10-06T02:46:25.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.409+0000 7f8102f67700 1 -- 172.21.15.33:0/3549866516 wait complete. 2023-10-06T02:46:25.429 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T02:46:25.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T02:46:25.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 Processor -- start 2023-10-06T02:46:25.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 -- start start 2023-10-06T02:46:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f057c0cca80 0x7f057c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f057c15c5b0 0x7f057c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057c0cf430 con 0x7f057c0cf8c0 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c0cfd40 con 0x7f057c15c5b0 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0582ebf700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c0ce820 con 0x7f057c0cca80 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f058145c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f057c15c5b0 0x7f057c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f057bfff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f057c0cf8c0 0x7f057c15bea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60730/0 (socket says 172.21.15.33:60730) 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f057bfff700 1 -- 172.21.15.33:0/1532074995 learned_addr learned my addr 172.21.15.33:0/1532074995 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0580c5b700 1 --2- 172.21.15.33:0/1532074995 >> v2:172.21.15.167:3300/0 conn(0x7f057c0cca80 0x7f057c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f057b7fe700 1 -- 172.21.15.33:0/1532074995 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69503434 0 0) 0x7f057c0cf430 con 0x7f057c0cf8c0 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f057b7fe700 1 -- 172.21.15.33:0/1532074995 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0564003570 con 0x7f057c0cf8c0 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f058145c700 1 -- 172.21.15.33:0/1532074995 >> v2:172.21.15.167:3300/0 conn(0x7f057c0cca80 msgr2=0x7f057c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f058145c700 1 --2- 172.21.15.33:0/1532074995 >> v2:172.21.15.167:3300/0 conn(0x7f057c0cca80 0x7f057c0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f058145c700 1 -- 172.21.15.33:0/1532074995 >> v1:172.21.15.33:6789/0 conn(0x7f057c0cf8c0 legacy=0x7f057c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f058145c700 1 -- 172.21.15.33:0/1532074995 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c0ceb00 con 0x7f057c15c5b0 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.537+0000 7f0580c5b700 1 --2- 172.21.15.33:0/1532074995 >> v2:172.21.15.167:3300/0 conn(0x7f057c0cca80 0x7f057c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f058145c700 1 --2- 172.21.15.33:0/1532074995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c15c5b0 0x7f057c0cdaf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f057c15dc20 tx=0x7f0574009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8642fea917eaa35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:25.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f057b7fe700 1 -- 172.21.15.33:0/1532074995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400e070 con 0x7f057c15c5b0 2023-10-06T02:46:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f057b7fe700 1 -- 172.21.15.33:0/1532074995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057400bb60 con 0x7f057c15c5b0 2023-10-06T02:46:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f057b7fe700 1 -- 172.21.15.33:0/1532074995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574012430 con 0x7f057c15c5b0 2023-10-06T02:46:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/1532074995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c15c5b0 msgr2=0x7f057c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/1532074995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c15c5b0 0x7f057c0cdaf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f057c15dc20 tx=0x7f0574009780 comp rx=0 tx=0).stop 2023-10-06T02:46:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/1532074995 shutdown_connections 2023-10-06T02:46:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/1532074995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c15c5b0 0x7f057c0cdaf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/1532074995 >> v2:172.21.15.167:3300/0 conn(0x7f057c0cca80 0x7f057c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/1532074995 >> 172.21.15.33:0/1532074995 conn(0x7f057c067a40 msgr2=0x7f057c068690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/1532074995 shutdown_connections 2023-10-06T02:46:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/1532074995 wait complete. 2023-10-06T02:46:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 Processor -- start 2023-10-06T02:46:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.541+0000 7f0582ebf700 1 -- start start 2023-10-06T02:46:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 0x7f057c068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f057c0cf8c0 0x7f057c14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f057c15c5b0 0x7f057c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c15fa50 con 0x7f057c0cca80 2023-10-06T02:46:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c15ee60 con 0x7f057c15c5b0 2023-10-06T02:46:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c15f120 con 0x7f057c0cf8c0 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 0x7f057c068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 0x7f057c068a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35892/0 (socket says 172.21.15.33:35892) 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 -- 172.21.15.33:0/3629885147 learned_addr learned my addr 172.21.15.33:0/3629885147 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f057bfff700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f057c0cf8c0 0x7f057c14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f058145c700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f057c15c5b0 0x7f057c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 -- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f057c0cf8c0 msgr2=0x7f057c14f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f057c0cf8c0 0x7f057c14f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 -- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f057c15c5b0 msgr2=0x7f057c14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f057c15c5b0 0x7f057c14fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 -- 172.21.15.33:0/3629885147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574009a20 con 0x7f057c0cca80 2023-10-06T02:46:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f058145c700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f057c15c5b0 0x7f057c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0580c5b700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 0x7f057c068a00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f056c00e8d0 tx=0x7f056c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00ee70 con 0x7f057c0cca80 2023-10-06T02:46:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c00c5e0 con 0x7f057c0cca80 2023-10-06T02:46:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00c980 con 0x7f057c0cca80 2023-10-06T02:46:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c15f400 con 0x7f057c0cca80 2023-10-06T02:46:25.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.545+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c068f40 con 0x7f057c0cca80 2023-10-06T02:46:25.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.549+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c0704c0 con 0x7f057c0cca80 2023-10-06T02:46:25.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.549+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f056c013410 con 0x7f057c0cca80 2023-10-06T02:46:25.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.549+0000 7f0578ff9700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0568060550 0x7f0568062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:25.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.549+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5526+0+0 (secure 0 0 0) 0x7f056c07c540 con 0x7f057c0cca80 2023-10-06T02:46:25.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.549+0000 7f057bfff700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0568060550 0x7f0568062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:25.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.553+0000 7f057bfff700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0568060550 0x7f0568062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f057c16ccc0 tx=0x7f0570009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:25.564 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.557+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f056c04cbe0 con 0x7f057c0cca80 2023-10-06T02:46:25.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:25.717+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f057c0cede0 con 0x7f057c0cca80 2023-10-06T02:46:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.529+0000 7f0578ff9700 1 -- 172.21.15.33:0/3629885147 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f056c050200 con 0x7f057c0cca80 2023-10-06T02:46:26.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0568060550 msgr2=0x7f0568062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0568060550 0x7f0568062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f057c16ccc0 tx=0x7f0570009040 comp rx=0 tx=0).stop 2023-10-06T02:46:26.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 msgr2=0x7f057c068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 0x7f057c068a00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f056c00e8d0 tx=0x7f056c008040 comp rx=0 tx=0).stop 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 shutdown_connections 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0568060550 0x7f0568062a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f057c0cf8c0 0x7f057c14f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f057c15c5b0 0x7f057c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 --2- 172.21.15.33:0/3629885147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f057c0cca80 0x7f057c068a00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 >> 172.21.15.33:0/3629885147 conn(0x7f057c067a40 msgr2=0x7f057c157670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 shutdown_connections 2023-10-06T02:46:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.541+0000 7f0582ebf700 1 -- 172.21.15.33:0/3629885147 wait complete. 2023-10-06T02:46:26.549 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2023-10-06T02:46:26.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T02:46:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.677+0000 7f7af09f2700 1 Processor -- start 2023-10-06T02:46:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.677+0000 7f7af09f2700 1 -- start start 2023-10-06T02:46:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.677+0000 7f7af09f2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7aec0af3a0 0x7f7aec0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.677+0000 7f7af09f2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec0cabb0 con 0x7f7aec1589f0 2023-10-06T02:46:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec0afe20 con 0x7f7aec0685b0 2023-10-06T02:46:26.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec0b0100 con 0x7f7aec0af3a0 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7aec0af3a0 0x7f7aec0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7aec0af3a0 0x7f7aec0af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35902/0 (socket says 172.21.15.33:35902) 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7aea59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7aec1589f0 0x7f7aec158dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60734/0 (socket says 172.21.15.33:60734) 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/758278055 learned_addr learned my addr 172.21.15.33:0/758278055 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/758278055 >> v1:172.21.15.167:6789/0 conn(0x7f7aec0685b0 legacy=0x7f7aec162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/758278055 >> v1:172.21.15.33:6789/0 conn(0x7f7aec1589f0 legacy=0x7f7aec158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/758278055 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0b03e0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae9d9b700 1 --2- 172.21.15.33:0/758278055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec0af3a0 0x7f7aec0af7b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0099b0 tx=0x7f7adc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c5b32cde0efb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae959a700 1 -- 172.21.15.33:0/758278055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00b860 con 0x7f7aec0af3a0 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae959a700 1 -- 172.21.15.33:0/758278055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc00b9c0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7ae959a700 1 -- 172.21.15.33:0/758278055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00bd60 con 0x7f7aec0af3a0 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- 172.21.15.33:0/758278055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec0af3a0 msgr2=0x7f7aec0af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 --2- 172.21.15.33:0/758278055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec0af3a0 0x7f7aec0af7b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0099b0 tx=0x7f7adc009250 comp rx=0 tx=0).stop 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- 172.21.15.33:0/758278055 shutdown_connections 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 --2- 172.21.15.33:0/758278055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec0af3a0 0x7f7aec0af7b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- 172.21.15.33:0/758278055 >> 172.21.15.33:0/758278055 conn(0x7f7aec066fb0 msgr2=0x7f7aec0673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- 172.21.15.33:0/758278055 shutdown_connections 2023-10-06T02:46:26.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.681+0000 7f7af09f2700 1 -- 172.21.15.33:0/758278055 wait complete. 2023-10-06T02:46:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 Processor -- start 2023-10-06T02:46:26.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 -- start start 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aec0685b0 0x7f7aec156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aec0af3a0 0x7f7aec169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 0x7f7aec15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec15b310 con 0x7f7aec1589f0 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0c3a30 con 0x7f7aec0af3a0 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7af09f2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0c3d10 con 0x7f7aec0685b0 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7aead9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 0x7f7aec15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7aead9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 0x7f7aec15a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35910/0 (socket says 172.21.15.33:35910) 2023-10-06T02:46:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7aead9d700 1 -- 172.21.15.33:0/1182162876 learned_addr learned my addr 172.21.15.33:0/1182162876 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:26.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7ae9d9b700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aec0af3a0 0x7f7aec169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:26.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7aea59c700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aec0685b0 0x7f7aec156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aec0685b0 msgr2=0x7f7aec156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7ae9d9b700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aec0685b0 0x7f7aec156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/1182162876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 msgr2=0x7f7aec15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7ae9d9b700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 0x7f7aec15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7ae9d9b700 1 -- 172.21.15.33:0/1182162876 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0094f0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.685+0000 7f7aead9d700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 0x7f7aec15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:26.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ae9d9b700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aec0af3a0 0x7f7aec169630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7aec064100 tx=0x7f7adc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc024440 con 0x7f7aec0af3a0 2023-10-06T02:46:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc0245a0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec0c3ff0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc024940 con 0x7f7aec0af3a0 2023-10-06T02:46:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec1567a0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec0c44f0 con 0x7f7aec0af3a0 2023-10-06T02:46:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7adc01a420 con 0x7f7aec0af3a0 2023-10-06T02:46:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ada7fc700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7acc060550 0x7f7acc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:26.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.689+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5896+0+0 (secure 0 0 0) 0x7f7adc07e680 con 0x7f7aec0af3a0 2023-10-06T02:46:26.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.693+0000 7f7aea59c700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7acc060550 0x7f7acc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:26.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.693+0000 7f7aea59c700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7acc060550 0x7f7acc062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0004fd0 tx=0x7f7ae0007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:26.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.701+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7adc04ec40 con 0x7f7aec0af3a0 2023-10-06T02:46:26.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:26.873+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7aec0710b0 con 0x7f7aec0af3a0 2023-10-06T02:46:27.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.549+0000 7f7ada7fc700 1 -- 172.21.15.33:0/1182162876 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7adc07ce60 con 0x7f7aec0af3a0 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7acc060550 msgr2=0x7f7acc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7acc060550 0x7f7acc062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0004fd0 tx=0x7f7ae0007eb0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aec0af3a0 msgr2=0x7f7aec169630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aec0af3a0 0x7f7aec169630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7aec064100 tx=0x7f7adc017000 comp rx=0 tx=0).stop 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 shutdown_connections 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec1589f0 0x7f7aec15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aec0af3a0 0x7f7aec169630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7acc060550 0x7f7acc062a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 --2- 172.21.15.33:0/1182162876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7aec0685b0 0x7f7aec156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 >> 172.21.15.33:0/1182162876 conn(0x7f7aec066fb0 msgr2=0x7f7aec157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 shutdown_connections 2023-10-06T02:46:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.569+0000 7f7af09f2700 1 -- 172.21.15.33:0/1182162876 wait complete. 2023-10-06T02:46:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2023-10-06T02:46:27.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T02:46:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.689+0000 7f748b590700 1 Processor -- start 2023-10-06T02:46:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.689+0000 7f748b590700 1 -- start start 2023-10-06T02:46:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7484159030 0x7f74840caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74840cb020 0x7f74840ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840cd7f0 con 0x7f7484158c60 2023-10-06T02:46:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840cddb0 con 0x7f74840cb020 2023-10-06T02:46:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7484159480 con 0x7f7484159030 2023-10-06T02:46:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74840cb020 0x7f74840ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74840cb020 0x7f74840ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35914/0 (socket says 172.21.15.33:35914) 2023-10-06T02:46:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 -- 172.21.15.33:0/2772564754 learned_addr learned my addr 172.21.15.33:0/2772564754 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f7489d8d700 1 --2- 172.21.15.33:0/2772564754 >> v2:172.21.15.167:3300/0 conn(0x7f7484159030 0x7f74840caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748958c700 1 -- 172.21.15.33:0/2772564754 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825911539 0 0) 0x7f74840cd7f0 con 0x7f7484158c60 2023-10-06T02:46:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748958c700 1 -- 172.21.15.33:0/2772564754 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7470003570 con 0x7f7484158c60 2023-10-06T02:46:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 -- 172.21.15.33:0/2772564754 >> v2:172.21.15.167:3300/0 conn(0x7f7484159030 msgr2=0x7f74840caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 --2- 172.21.15.33:0/2772564754 >> v2:172.21.15.167:3300/0 conn(0x7f7484159030 0x7f74840caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 -- 172.21.15.33:0/2772564754 >> v1:172.21.15.167:6789/0 conn(0x7f7484158c60 legacy=0x7f748415cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 -- 172.21.15.33:0/2772564754 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748415d630 con 0x7f74840cb020 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748ad8f700 1 --2- 172.21.15.33:0/2772564754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f74840ce240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7484088430 tx=0x7f747c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e24c83e491aa09c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748958c700 1 -- 172.21.15.33:0/2772564754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00b640 con 0x7f74840cb020 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748958c700 1 -- 172.21.15.33:0/2772564754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00b7a0 con 0x7f74840cb020 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748958c700 1 -- 172.21.15.33:0/2772564754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00bb60 con 0x7f74840cb020 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- 172.21.15.33:0/2772564754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 msgr2=0x7f74840ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 --2- 172.21.15.33:0/2772564754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f74840ce240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7484088430 tx=0x7f747c009250 comp rx=0 tx=0).stop 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- 172.21.15.33:0/2772564754 shutdown_connections 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 --2- 172.21.15.33:0/2772564754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f74840ce240 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 --2- 172.21.15.33:0/2772564754 >> v2:172.21.15.167:3300/0 conn(0x7f7484159030 0x7f74840caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- 172.21.15.33:0/2772564754 >> 172.21.15.33:0/2772564754 conn(0x7f7484067a80 msgr2=0x7f74840cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- 172.21.15.33:0/2772564754 shutdown_connections 2023-10-06T02:46:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.693+0000 7f748b590700 1 -- 172.21.15.33:0/2772564754 wait complete. 2023-10-06T02:46:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 Processor -- start 2023-10-06T02:46:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 -- start start 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f748416c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7484158c60 0x7f74840c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7484159030 0x7f74841506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748416d410 con 0x7f74840cb020 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840b1610 con 0x7f7484159030 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748b590700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74840b0a00 con 0x7f7484158c60 2023-10-06T02:46:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748a58e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f748416c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748a58e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f748416c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35924/0 (socket says 172.21.15.33:35924) 2023-10-06T02:46:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748a58e700 1 -- 172.21.15.33:0/4264788875 learned_addr learned my addr 172.21.15.33:0/4264788875 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f7489d8d700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7484158c60 0x7f74840c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.697+0000 7f748ad8f700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7484159030 0x7f74841506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f7489d8d700 1 -- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7484159030 msgr2=0x7f74841506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f7489d8d700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7484159030 0x7f74841506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f7489d8d700 1 -- 172.21.15.33:0/4264788875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 msgr2=0x7f748416c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f7489d8d700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f748416c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f7489d8d700 1 -- 172.21.15.33:0/4264788875 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0094f0 con 0x7f7484158c60 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f748a58e700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f748416c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f7489d8d700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7484158c60 0x7f74840c6cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f747400e900 tx=0x7f74740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474013680 con 0x7f7484158c60 2023-10-06T02:46:27.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74740137e0 con 0x7f7484158c60 2023-10-06T02:46:27.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474013ba0 con 0x7f7484158c60 2023-10-06T02:46:27.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74840b0ce0 con 0x7f7484158c60 2023-10-06T02:46:27.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74840b12a0 con 0x7f7484158c60 2023-10-06T02:46:27.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74840b07d0 con 0x7f7484158c60 2023-10-06T02:46:27.706 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f74740123d0 con 0x7f7484158c60 2023-10-06T02:46:27.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.701+0000 7f747a7fc700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7468060550 0x7f7468062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:27.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.705+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6278+0+0 (secure 0 0 0) 0x7f747407cbe0 con 0x7f7484158c60 2023-10-06T02:46:27.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.705+0000 7f748a58e700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7468060550 0x7f7468062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:27.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.705+0000 7f748a58e700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7468060550 0x7f7468062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7480007fd0 tx=0x7f7480002820 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:27.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.713+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f747404cef0 con 0x7f7484158c60 2023-10-06T02:46:27.924 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:27.921+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7484071040 con 0x7f7484158c60 2023-10-06T02:46:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.557+0000 7f747a7fc700 1 -- 172.21.15.33:0/4264788875 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7474050510 con 0x7f7484158c60 2023-10-06T02:46:28.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7468060550 msgr2=0x7f7468062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7468060550 0x7f7468062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7480007fd0 tx=0x7f7480002820 comp rx=0 tx=0).stop 2023-10-06T02:46:28.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7484158c60 msgr2=0x7f74840c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7484158c60 0x7f74840c6cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f747400e900 tx=0x7f74740091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 shutdown_connections 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7468060550 0x7f7468062a00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7484158c60 0x7f74840c6cd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7484159030 0x7f74841506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 --2- 172.21.15.33:0/4264788875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840cb020 0x7f748416c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 >> 172.21.15.33:0/4264788875 conn(0x7f7484067a80 msgr2=0x7f74840ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 shutdown_connections 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.569+0000 7f748b590700 1 -- 172.21.15.33:0/4264788875 wait complete. 2023-10-06T02:46:28.574 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 6 and data pool 7 2023-10-06T02:46:28.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe597b73700 1 Processor -- start 2023-10-06T02:46:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe597b73700 1 -- start start 2023-10-06T02:46:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe597b73700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe590158bf0 0x7fe5900b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe597b73700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59015d830 con 0x7fe5900b0530 2023-10-06T02:46:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe597b73700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5900b00a0 con 0x7fe590158fc0 2023-10-06T02:46:28.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe597b73700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe590067c10 con 0x7fe590158bf0 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59510e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe5900b0530 0x7fe59015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60738/0 (socket says 172.21.15.33:60738) 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59510e700 1 -- 172.21.15.33:0/3235159435 learned_addr learned my addr 172.21.15.33:0/3235159435 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59590f700 1 --2- 172.21.15.33:0/3235159435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe590158bf0 0x7fe5900b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005813602 0 0) 0x7fe59015d830 con 0x7fe5900b0530 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe57c003570 con 0x7fe5900b0530 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893018267 0 0) 0x7fe5900b00a0 con 0x7fe590158fc0 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59015d830 con 0x7fe590158fc0 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59590f700 1 -- 172.21.15.33:0/3235159435 >> v1:172.21.15.167:6789/0 conn(0x7fe590158fc0 legacy=0x7fe5900ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59590f700 1 -- 172.21.15.33:0/3235159435 >> v1:172.21.15.33:6789/0 conn(0x7fe5900b0530 legacy=0x7fe59015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.685+0000 7fe59590f700 1 -- 172.21.15.33:0/3235159435 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900cba20 con 0x7fe590158bf0 2023-10-06T02:46:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59590f700 1 --2- 172.21.15.33:0/3235159435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe590158bf0 0x7fe5900b1b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe588009e90 tx=0x7fe588009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2120678328569a25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800e070 con 0x7fe590158bf0 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58800bb80 con 0x7fe590158bf0 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59490d700 1 -- 172.21.15.33:0/3235159435 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588012430 con 0x7fe590158bf0 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- 172.21.15.33:0/3235159435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe590158bf0 msgr2=0x7fe5900b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 --2- 172.21.15.33:0/3235159435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe590158bf0 0x7fe5900b1b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe588009e90 tx=0x7fe588009780 comp rx=0 tx=0).stop 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- 172.21.15.33:0/3235159435 shutdown_connections 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 --2- 172.21.15.33:0/3235159435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe590158bf0 0x7fe5900b1b90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- 172.21.15.33:0/3235159435 >> 172.21.15.33:0/3235159435 conn(0x7fe5900cfd70 msgr2=0x7fe5900d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- 172.21.15.33:0/3235159435 shutdown_connections 2023-10-06T02:46:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- 172.21.15.33:0/3235159435 wait complete. 2023-10-06T02:46:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 Processor -- start 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- start start 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 0x7fe59014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe590158fc0 0x7fe5900b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe59015aa70 0x7fe59016d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59016c9a0 con 0x7fe590158bf0 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59016cc80 con 0x7fe590158fc0 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe597b73700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59015aee0 con 0x7fe59015aa70 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59590f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 0x7fe59014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59590f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 0x7fe59014fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35940/0 (socket says 172.21.15.33:35940) 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.689+0000 7fe59590f700 1 -- 172.21.15.33:0/3384994327 learned_addr learned my addr 172.21.15.33:0/3384994327 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:28.696 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59510e700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe590158fc0 0x7fe5900b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe596110700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe59015aa70 0x7fe59016d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59590f700 1 -- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe59015aa70 msgr2=0x7fe59016d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59590f700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe59015aa70 0x7fe59016d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59590f700 1 -- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe590158fc0 msgr2=0x7fe5900b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59590f700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe590158fc0 0x7fe5900b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59590f700 1 -- 172.21.15.33:0/3384994327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588009a20 con 0x7fe590158bf0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe596110700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe59015aa70 0x7fe59016d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe59590f700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 0x7fe59014fd90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5880097c0 tx=0x7fe588019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800be40 con 0x7fe590158bf0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5880263f0 con 0x7fe590158bf0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59015b1c0 con 0x7fe590158bf0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880267b0 con 0x7fe590158bf0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5900b4220 con 0x7fe590158bf0 2023-10-06T02:46:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5900710b0 con 0x7fe590158bf0 2023-10-06T02:46:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.693+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe58802f450 con 0x7fe590158bf0 2023-10-06T02:46:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.697+0000 7fe585ffb700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe574060620 0x7fe574062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.697+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe58807e4d0 con 0x7fe590158bf0 2023-10-06T02:46:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.697+0000 7fe59510e700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe574060620 0x7fe574062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.697+0000 7fe59510e700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe574060620 0x7fe574062ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe5900b4530 tx=0x7fe580009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:28.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.705+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe58804e8a0 con 0x7fe590158bf0 2023-10-06T02:46:28.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5900d0660 con 0x7fe590158bf0 2023-10-06T02:46:28.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe585ffb700 1 -- 172.21.15.33:0/3384994327 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17084 (secure 0 0 0) 0x7fe588051ec0 con 0x7fe590158bf0 2023-10-06T02:46:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:28.873 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":43,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:27.926087+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:46:25.764531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T02:46:26.918595+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe574060620 msgr2=0x7fe574062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe574060620 0x7fe574062ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe5900b4530 tx=0x7fe580009040 comp rx=0 tx=0).stop 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 msgr2=0x7fe59014fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 0x7fe59014fd90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe5880097c0 tx=0x7fe588019000 comp rx=0 tx=0).stop 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 shutdown_connections 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe574060620 0x7fe574062ad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe59015aa70 0x7fe59016d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe590158fc0 0x7fe5900b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.869+0000 7fe597b73700 1 --2- 172.21.15.33:0/3384994327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe590158bf0 0x7fe59014fd90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.873+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 >> 172.21.15.33:0/3384994327 conn(0x7fe5900cfd70 msgr2=0x7fe5900685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.873+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 shutdown_connections 2023-10-06T02:46:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.873+0000 7fe597b73700 1 -- 172.21.15.33:0/3384994327 wait complete. 2023-10-06T02:46:28.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:28.992 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 Processor -- start 2023-10-06T02:46:28.993 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- start start 2023-10-06T02:46:28.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1fe00cf420 0x7f1fe015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.996 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe0068b60 0x7f1fe015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe015d4f0 con 0x7f1fe0158d00 2023-10-06T02:46:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe015da10 con 0x7f1fe00cf420 2023-10-06T02:46:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00cdf80 con 0x7f1fe0068b60 2023-10-06T02:46:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1fe00cf420 0x7f1fe015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1fe00cf420 0x7f1fe015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35942/0 (socket says 172.21.15.33:35942) 2023-10-06T02:46:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 -- 172.21.15.33:0/3590610168 learned_addr learned my addr 172.21.15.33:0/3590610168 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5d1a700 1 --1- 172.21.15.33:0/3590610168 >> v1:172.21.15.33:6789/0 conn(0x7f1fe0158d00 0x7f1fe0068450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60750/0 (socket says 172.21.15.33:60750) 2023-10-06T02:46:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe651b700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe0068b60 0x7f1fe015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 -- 172.21.15.33:0/3590610168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe0068b60 msgr2=0x7f1fe015cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe0068b60 0x7f1fe015cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 -- 172.21.15.33:0/3590610168 >> v1:172.21.15.33:6789/0 conn(0x7f1fe0158d00 legacy=0x7f1fe0068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 -- 172.21.15.33:0/3590610168 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe015d730 con 0x7f1fe00cf420 2023-10-06T02:46:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe651b700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe0068b60 0x7f1fe015cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe5519700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe00cf420 0x7f1fe015ca70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009f20 tx=0x7f1fd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cb2890767c5763 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe4d18700 1 -- 172.21.15.33:0/3590610168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800e070 con 0x7f1fe00cf420 2023-10-06T02:46:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe4d18700 1 -- 172.21.15.33:0/3590610168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd800ba80 con 0x7f1fe00cf420 2023-10-06T02:46:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe4d18700 1 -- 172.21.15.33:0/3590610168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80092e0 con 0x7f1fe00cf420 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3590610168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe00cf420 msgr2=0x7f1fe015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe00cf420 0x7f1fe015ca70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009f20 tx=0x7f1fd8009840 comp rx=0 tx=0).stop 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3590610168 shutdown_connections 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe0068b60 0x7f1fe015cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3590610168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe00cf420 0x7f1fe015ca70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3590610168 >> 172.21.15.33:0/3590610168 conn(0x7f1fe0067a80 msgr2=0x7f1fe00cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:29.000 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3590610168 shutdown_connections 2023-10-06T02:46:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.989+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3590610168 wait complete. 2023-10-06T02:46:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 Processor -- start 2023-10-06T02:46:29.001 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 -- start start 2023-10-06T02:46:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe0068b60 0x7f1fe016a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe00cf420 0x7f1fe0151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 0x7f1fe00b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe014a400 con 0x7f1fe0158d00 2023-10-06T02:46:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe01524e0 con 0x7f1fe0068b60 2023-10-06T02:46:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe014bd10 con 0x7f1fe00cf420 2023-10-06T02:46:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 0x7f1fe00b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 0x7f1fe00b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35954/0 (socket says 172.21.15.33:35954) 2023-10-06T02:46:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 -- 172.21.15.33:0/3294932593 learned_addr learned my addr 172.21.15.33:0/3294932593 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe5d1a700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe0068b60 0x7f1fe016a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe5519700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe00cf420 0x7f1fe0151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 -- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe00cf420 msgr2=0x7f1fe0151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe00cf420 0x7f1fe0151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 -- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe0068b60 msgr2=0x7f1fe016a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe0068b60 0x7f1fe016a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 -- 172.21.15.33:0/3294932593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009ae0 con 0x7f1fe0158d00 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe651b700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 0x7f1fe00b5260 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c880 tx=0x7f1fd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0011600 con 0x7f1fe0158d00 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0011760 con 0x7f1fe0158d00 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0011b20 con 0x7f1fe0158d00 2023-10-06T02:46:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe014bf60 con 0x7f1fe0158d00 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.993+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe00b57a0 con 0x7f1fe0158d00 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.997+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe0071040 con 0x7f1fe0158d00 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.997+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1fd00193b0 con 0x7f1fe0158d00 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.997+0000 7f1fddffb700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fc8060580 0x7f1fc8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.997+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1fd007c660 con 0x7f1fe0158d00 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:28.997+0000 7f1fe5d1a700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fc8060580 0x7f1fc8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.001+0000 7f1fe5d1a700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fc8060580 0x7f1fc8062a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc004fd0 tx=0x7f1fcc007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.011 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.005+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1fd00504f0 con 0x7f1fe0158d00 2023-10-06T02:46:29.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.169+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fe00cd930 con 0x7f1fe0158d00 2023-10-06T02:46:29.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.169+0000 7f1fddffb700 1 -- 172.21.15.33:0/3294932593 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f1fd0050080 con 0x7f1fe0158d00 2023-10-06T02:46:29.175 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:29.175 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":43,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:27.926087+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:46:25.764531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T02:46:26.918595+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:29.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fc8060580 msgr2=0x7f1fc8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fc8060580 0x7f1fc8062a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc004fd0 tx=0x7f1fcc007eb0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 msgr2=0x7f1fe00b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 0x7f1fe00b5260 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c880 tx=0x7f1fd0008040 comp rx=0 tx=0).stop 2023-10-06T02:46:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 shutdown_connections 2023-10-06T02:46:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1fc8060580 0x7f1fc8062a30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe00cf420 0x7f1fe0151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fe0158d00 0x7f1fe00b5260 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 --2- 172.21.15.33:0/3294932593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe0068b60 0x7f1fe016a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 >> 172.21.15.33:0/3294932593 conn(0x7f1fe0067a80 msgr2=0x7f1fe014ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:29.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 shutdown_connections 2023-10-06T02:46:29.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.173+0000 7f1fe6d1c700 1 -- 172.21.15.33:0/3294932593 wait complete. 2023-10-06T02:46:29.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T02:46:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.309+0000 7fa0dcbce700 1 Processor -- start 2023-10-06T02:46:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.309+0000 7fa0dcbce700 1 -- start start 2023-10-06T02:46:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0dcbce700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa0d8155710 0x7fa0d80c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0dcbce700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d80c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0dcbce700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d8159c70 con 0x7fa0d80cbbc0 2023-10-06T02:46:29.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0dcbce700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d80c9100 con 0x7fa0d8155710 2023-10-06T02:46:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0dcbce700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80c8e20 con 0x7fa0d815a100 2023-10-06T02:46:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa0d8155710 0x7fa0d80c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa0d80cbbc0 0x7fa0d80ca4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60756/0 (socket says 172.21.15.33:60756) 2023-10-06T02:46:29.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d7fff700 1 -- 172.21.15.33:0/4093081288 learned_addr learned my addr 172.21.15.33:0/4093081288 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d6ffd700 1 --2- 172.21.15.33:0/4093081288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d80c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d67fc700 1 -- 172.21.15.33:0/4093081288 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138672550 0 0) 0x7fa0d8159c70 con 0x7fa0d80cbbc0 2023-10-06T02:46:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d67fc700 1 -- 172.21.15.33:0/4093081288 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c4003570 con 0x7fa0d80cbbc0 2023-10-06T02:46:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/4093081288 >> v2:172.21.15.33:3300/0 conn(0x7fa0d8155710 msgr2=0x7fa0d80c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d6ffd700 1 --2- 172.21.15.33:0/4093081288 >> v2:172.21.15.33:3300/0 conn(0x7fa0d8155710 0x7fa0d80c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/4093081288 >> v1:172.21.15.33:6789/0 conn(0x7fa0d80cbbc0 legacy=0x7fa0d80ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/4093081288 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d8159620 con 0x7fa0d815a100 2023-10-06T02:46:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d77fe700 1 --2- 172.21.15.33:0/4093081288 >> v2:172.21.15.33:3300/0 conn(0x7fa0d8155710 0x7fa0d80c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:29.320 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.313+0000 7fa0d6ffd700 1 --2- 172.21.15.33:0/4093081288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d80c8810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80098e0 tx=0x7fa0c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183ac91915e646bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0d67fc700 1 -- 172.21.15.33:0/4093081288 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8011630 con 0x7fa0d815a100 2023-10-06T02:46:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0d67fc700 1 -- 172.21.15.33:0/4093081288 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8011790 con 0x7fa0d815a100 2023-10-06T02:46:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0d67fc700 1 -- 172.21.15.33:0/4093081288 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8011b30 con 0x7fa0d815a100 2023-10-06T02:46:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 -- 172.21.15.33:0/4093081288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 msgr2=0x7fa0d80c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/4093081288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d80c8810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80098e0 tx=0x7fa0c80091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 -- 172.21.15.33:0/4093081288 shutdown_connections 2023-10-06T02:46:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/4093081288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d80c8810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/4093081288 >> v2:172.21.15.33:3300/0 conn(0x7fa0d8155710 0x7fa0d80c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 -- 172.21.15.33:0/4093081288 >> 172.21.15.33:0/4093081288 conn(0x7fa0d80c9350 msgr2=0x7fa0d80c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 -- 172.21.15.33:0/4093081288 shutdown_connections 2023-10-06T02:46:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 -- 172.21.15.33:0/4093081288 wait complete. 2023-10-06T02:46:29.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 Processor -- start 2023-10-06T02:46:29.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.317+0000 7fa0dcbce700 1 -- start start 2023-10-06T02:46:29.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d80cbbc0 0x7fa0d80ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 0x7fa0d816ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d816b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80b1010 con 0x7fa0d8155710 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80b0400 con 0x7fa0d80cbbc0 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80b06e0 con 0x7fa0d815a100 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 0x7fa0d816ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 0x7fa0d816ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35984/0 (socket says 172.21.15.33:35984) 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/2411378513 learned_addr learned my addr 172.21.15.33:0/2411378513 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d77fe700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d80cbbc0 0x7fa0d80ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d7fff700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d816b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 msgr2=0x7fa0d816b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d816b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d80cbbc0 msgr2=0x7fa0d80ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d80cbbc0 0x7fa0d80ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 -- 172.21.15.33:0/2411378513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c0009420 con 0x7fa0d8155710 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0d6ffd700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 0x7fa0d816ad00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80c8f60 tx=0x7fa0c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8011900 con 0x7fa0d8155710 2023-10-06T02:46:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8011aa0 con 0x7fa0d8155710 2023-10-06T02:46:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8025640 con 0x7fa0d8155710 2023-10-06T02:46:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c8009420 con 0x7fa0d8155710 2023-10-06T02:46:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.321+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d80b0ca0 con 0x7fa0d8155710 2023-10-06T02:46:29.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.325+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d80c3970 con 0x7fa0d8155710 2023-10-06T02:46:29.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.325+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa0c8025a30 con 0x7fa0d8155710 2023-10-06T02:46:29.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.325+0000 7fa0cf7fe700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa0b8060550 0x7fa0b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.325+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa0c807e000 con 0x7fa0d8155710 2023-10-06T02:46:29.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.325+0000 7fa0d77fe700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa0b8060550 0x7fa0b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.325+0000 7fa0d77fe700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa0b8060550 0x7fa0b8062a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009f90 tx=0x7fa0c00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.329+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa0c804e350 con 0x7fa0d8155710 2023-10-06T02:46:29.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.529+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa0d8071040 con 0x7fa0d8155710 2023-10-06T02:46:29.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.565+0000 7fa0cf7fe700 1 -- 172.21.15.33:0/2411378513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa0c8051970 con 0x7fa0d8155710 2023-10-06T02:46:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa0b8060550 msgr2=0x7fa0b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa0b8060550 0x7fa0b8062a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009f90 tx=0x7fa0c00091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 msgr2=0x7fa0d816ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 0x7fa0d816ad00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80c8f60 tx=0x7fa0c8017000 comp rx=0 tx=0).stop 2023-10-06T02:46:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 shutdown_connections 2023-10-06T02:46:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa0b8060550 0x7fa0b8062a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0d815a100 0x7fa0d816b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0d8155710 0x7fa0d816ad00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 --2- 172.21.15.33:0/2411378513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d80cbbc0 0x7fa0d80ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 >> 172.21.15.33:0/2411378513 conn(0x7fa0d80c9350 msgr2=0x7fa0d806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 shutdown_connections 2023-10-06T02:46:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.569+0000 7fa0dcbce700 1 -- 172.21.15.33:0/2411378513 wait complete. 2023-10-06T02:46:29.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T02:46:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 Processor -- start 2023-10-06T02:46:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 -- start start 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 0x7f51b806a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 0x7f51b806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b806aff0 con 0x7f51b8154e20 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b80c7590 con 0x7f51b8066fb0 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b806b230 con 0x7f51b80cb180 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b67fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 0x7f51b806a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b77fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 0x7f51b806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b67fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 0x7f51b806a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41476/0 (socket says 172.21.15.33:41476) 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b77fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 0x7f51b806aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:44194/0 (socket says 172.21.15.33:44194) 2023-10-06T02:46:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b67fc700 1 -- 172.21.15.33:0/2893536757 learned_addr learned my addr 172.21.15.33:0/2893536757 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:29.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b67fc700 1 -- 172.21.15.33:0/2893536757 >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 msgr2=0x7f51b806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:29.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b67fc700 1 --2- 172.21.15.33:0/2893536757 >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 0x7f51b806aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.701+0000 7f51b67fc700 1 -- 172.21.15.33:0/2893536757 >> v1:172.21.15.167:6789/0 conn(0x7f51b8154e20 legacy=0x7f51b80caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b67fc700 1 -- 172.21.15.33:0/2893536757 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b806b510 con 0x7f51b80cb180 2023-10-06T02:46:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b77fe700 1 --2- 172.21.15.33:0/2893536757 >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 0x7f51b806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b67fc700 1 --2- 172.21.15.33:0/2893536757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 0x7f51b806a570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0099e0 tx=0x7f51ac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f656124bc041fcc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b5ffb700 1 -- 172.21.15.33:0/2893536757 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00b820 con 0x7f51b80cb180 2023-10-06T02:46:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b5ffb700 1 -- 172.21.15.33:0/2893536757 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac00b980 con 0x7f51b80cb180 2023-10-06T02:46:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b5ffb700 1 -- 172.21.15.33:0/2893536757 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00bd20 con 0x7f51b80cb180 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 -- 172.21.15.33:0/2893536757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 msgr2=0x7f51b806a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 --2- 172.21.15.33:0/2893536757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 0x7f51b806a570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0099e0 tx=0x7f51ac009250 comp rx=0 tx=0).stop 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 -- 172.21.15.33:0/2893536757 shutdown_connections 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 --2- 172.21.15.33:0/2893536757 >> v2:172.21.15.167:3300/0 conn(0x7f51b8066fb0 0x7f51b806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 --2- 172.21.15.33:0/2893536757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b80cb180 0x7f51b806a570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 -- 172.21.15.33:0/2893536757 >> 172.21.15.33:0/2893536757 conn(0x7f51b80c8ad0 msgr2=0x7f51b80c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:29.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 -- 172.21.15.33:0/2893536757 shutdown_connections 2023-10-06T02:46:29.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.705+0000 7f51b7fff700 1 -- 172.21.15.33:0/2893536757 wait complete. 2023-10-06T02:46:29.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 Processor -- start 2023-10-06T02:46:29.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 -- start start 2023-10-06T02:46:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8066fb0 0x7f51b814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b80cb180 0x7f51b80c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 0x7f51b8153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80c4a40 con 0x7f51b8154e20 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8154340 con 0x7f51b80cb180 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8154620 con 0x7f51b8066fb0 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 0x7f51b8153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 0x7f51b8153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35994/0 (socket says 172.21.15.33:35994) 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b77fe700 1 -- 172.21.15.33:0/1001739732 learned_addr learned my addr 172.21.15.33:0/1001739732 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b67fc700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b80cb180 0x7f51b80c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.709+0000 7f51b6ffd700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8066fb0 0x7f51b814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b77fe700 1 -- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8066fb0 msgr2=0x7f51b814cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b77fe700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8066fb0 0x7f51b814cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b77fe700 1 -- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b80cb180 msgr2=0x7f51b80c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b77fe700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b80cb180 0x7f51b80c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b77fe700 1 -- 172.21.15.33:0/1001739732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0009a20 con 0x7f51b8154e20 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b77fe700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 0x7f51b8153860 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009f30 tx=0x7f51b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b000cb70 con 0x7f51b8154e20 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b000f040 con 0x7f51b8154e20 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0014480 con 0x7f51b8154e20 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac0094f0 con 0x7f51b8154e20 2023-10-06T02:46:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b80c46d0 con 0x7f51b8154e20 2023-10-06T02:46:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b8071040 con 0x7f51b8154e20 2023-10-06T02:46:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f51b00147f0 con 0x7f51b8154e20 2023-10-06T02:46:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f519effd700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f51a4060550 0x7f51a4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.713+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f51b007d820 con 0x7f51b8154e20 2023-10-06T02:46:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.717+0000 7f51b6ffd700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f51a4060550 0x7f51a4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.717+0000 7f51b6ffd700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f51a4060550 0x7f51a4062a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51a8003950 tx=0x7f51a8007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.721+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f51b004dad0 con 0x7f51b8154e20 2023-10-06T02:46:29.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:29.917+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51b806b880 con 0x7f51b8154e20 2023-10-06T02:46:30.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.577+0000 7f519effd700 1 -- 172.21.15.33:0/1001739732 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51b00510f0 con 0x7f51b8154e20 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f51a4060550 msgr2=0x7f51a4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f51a4060550 0x7f51a4062a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51a8003950 tx=0x7f51a8007ec0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 msgr2=0x7f51b8153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 0x7f51b8153860 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009f30 tx=0x7f51b000b040 comp rx=0 tx=0).stop 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 shutdown_connections 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b8154e20 0x7f51b8153860 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b80cb180 0x7f51b80c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f51a4060550 0x7f51a4062a00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 --2- 172.21.15.33:0/1001739732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b8066fb0 0x7f51b814cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 >> 172.21.15.33:0/1001739732 conn(0x7f51b80c8ad0 msgr2=0x7f51b80c0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 shutdown_connections 2023-10-06T02:46:30.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.581+0000 7f51b7fff700 1 -- 172.21.15.33:0/1001739732 wait complete. 2023-10-06T02:46:30.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:30.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.709+0000 7efd9eaf9700 1 Processor -- start 2023-10-06T02:46:30.715 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.709+0000 7efd9eaf9700 1 -- start start 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd9806d690 0x7efd980b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd98069c10 0x7efd980b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd980b19a0 con 0x7efd98158b50 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd980b09e0 con 0x7efd98069c10 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd98069780 con 0x7efd9806d690 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd98069c10 0x7efd980b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd98069c10 0x7efd980b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36006/0 (socket says 172.21.15.33:36006) 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 -- 172.21.15.33:0/3951558407 learned_addr learned my addr 172.21.15.33:0/3951558407 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:30.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9d2f6700 1 --2- 172.21.15.33:0/3951558407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd9806d690 0x7efd980b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 -- 172.21.15.33:0/3951558407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd9806d690 msgr2=0x7efd980b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 --2- 172.21.15.33:0/3951558407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd9806d690 0x7efd980b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 -- 172.21.15.33:0/3951558407 >> v1:172.21.15.167:6789/0 conn(0x7efd98158b50 legacy=0x7efd9806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 -- 172.21.15.33:0/3951558407 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98069170 con 0x7efd98069c10 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9e2f8700 1 --2- 172.21.15.33:0/3951558407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd980b1460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efd90009f20 tx=0x7efd90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a444ceb4b1acd35d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9caf5700 1 -- 172.21.15.33:0/3951558407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9000e070 con 0x7efd98069c10 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9caf5700 1 -- 172.21.15.33:0/3951558407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9000ba60 con 0x7efd98069c10 2023-10-06T02:46:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9caf5700 1 -- 172.21.15.33:0/3951558407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900092e0 con 0x7efd98069c10 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- 172.21.15.33:0/3951558407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 msgr2=0x7efd980b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/3951558407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd980b1460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efd90009f20 tx=0x7efd90009840 comp rx=0 tx=0).stop 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- 172.21.15.33:0/3951558407 shutdown_connections 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/3951558407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd980b1460 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/3951558407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd9806d690 0x7efd980b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- 172.21.15.33:0/3951558407 >> 172.21.15.33:0/3951558407 conn(0x7efd9806c2d0 msgr2=0x7efd980af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- 172.21.15.33:0/3951558407 shutdown_connections 2023-10-06T02:46:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.713+0000 7efd9eaf9700 1 -- 172.21.15.33:0/3951558407 wait complete. 2023-10-06T02:46:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 Processor -- start 2023-10-06T02:46:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 -- start start 2023-10-06T02:46:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd981518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9806d690 0x7efd9815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd98158b50 0x7efd980c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815e3d0 con 0x7efd98069c10 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815df40 con 0x7efd9806d690 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9eaf9700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815d330 con 0x7efd98158b50 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.717+0000 7efd9daf7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd981518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd981518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36022/0 (socket says 172.21.15.33:36022) 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 -- 172.21.15.33:0/4256460127 learned_addr learned my addr 172.21.15.33:0/4256460127 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9d2f6700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9806d690 0x7efd9815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9e2f8700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd98158b50 0x7efd980c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 -- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd98158b50 msgr2=0x7efd980c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd98158b50 0x7efd980c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 -- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9806d690 msgr2=0x7efd9815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9806d690 0x7efd9815c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 -- 172.21.15.33:0/4256460127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd90009ae0 con 0x7efd98069c10 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9daf7700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd981518c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c590 tx=0x7efd94007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94016070 con 0x7efd98069c10 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94006820 con 0x7efd98069c10 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9815d610 con 0x7efd98069c10 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94006bc0 con 0x7efd98069c10 2023-10-06T02:46:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9815dbd0 con 0x7efd98069c10 2023-10-06T02:46:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd98071040 con 0x7efd98069c10 2023-10-06T02:46:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.721+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efd94006fd0 con 0x7efd98069c10 2023-10-06T02:46:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.725+0000 7efd8dffb700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efd7c060550 0x7efd7c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.725+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7efd9407c120 con 0x7efd98069c10 2023-10-06T02:46:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.725+0000 7efd9d2f6700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efd7c060550 0x7efd7c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.725+0000 7efd9d2f6700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efd7c060550 0x7efd7c062a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd9815ea90 tx=0x7efd880091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.729+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efd9404c470 con 0x7efd98069c10 2023-10-06T02:46:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd980b1bd0 con 0x7efd98069c10 2023-10-06T02:46:30.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd8dffb700 1 -- 172.21.15.33:0/4256460127 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7efd9404fa90 con 0x7efd98069c10 2023-10-06T02:46:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:30.929 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efd7c060550 msgr2=0x7efd7c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efd7c060550 0x7efd7c062a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd9815ea90 tx=0x7efd880091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 msgr2=0x7efd981518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd981518c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c590 tx=0x7efd94007f50 comp rx=0 tx=0).stop 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 shutdown_connections 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efd7c060550 0x7efd7c062a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd98158b50 0x7efd980c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9806d690 0x7efd9815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.925+0000 7efd9eaf9700 1 --2- 172.21.15.33:0/4256460127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd98069c10 0x7efd981518c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:30.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.929+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 >> 172.21.15.33:0/4256460127 conn(0x7efd9806c2d0 msgr2=0x7efd9815aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:30.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.929+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 shutdown_connections 2023-10-06T02:46:30.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:30.929+0000 7efd9eaf9700 1 -- 172.21.15.33:0/4256460127 wait complete. 2023-10-06T02:46:30.933 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:30.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 Processor -- start 2023-10-06T02:46:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 -- start start 2023-10-06T02:46:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdac0adb60 0x7fbdac0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac15ace0 0x7fbdac069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac157eb0 con 0x7fbdac067970 2023-10-06T02:46:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdac069820 con 0x7fbdac0adb60 2023-10-06T02:46:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdabfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac067ea0 con 0x7fbdac15ace0 2023-10-06T02:46:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdaaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdac0adb60 0x7fbdac0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdaaffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbdac0adb60 0x7fbdac0c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36034/0 (socket says 172.21.15.33:36034) 2023-10-06T02:46:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdaaffd700 1 -- 172.21.15.33:0/892942180 learned_addr learned my addr 172.21.15.33:0/892942180 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.041+0000 7fbdab7fe700 1 --2- 172.21.15.33:0/892942180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac15ace0 0x7fbdac069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdaaffd700 1 -- 172.21.15.33:0/892942180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac15ace0 msgr2=0x7fbdac069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdaaffd700 1 --2- 172.21.15.33:0/892942180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac15ace0 0x7fbdac069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdaaffd700 1 -- 172.21.15.33:0/892942180 >> v1:172.21.15.167:6789/0 conn(0x7fbdac067970 legacy=0x7fbdac15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbda9ffb700 1 -- 172.21.15.33:0/892942180 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293120258 0 0) 0x7fbd94001490 con 0x7fbdac067970 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdaaffd700 1 -- 172.21.15.33:0/892942180 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac068180 con 0x7fbdac0adb60 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdaaffd700 1 --2- 172.21.15.33:0/892942180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac0adb60 0x7fbdac0c6820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0099b0 tx=0x7fbd9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941c70d55df841af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbda9ffb700 1 -- 172.21.15.33:0/892942180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c009ea0 con 0x7fbdac0adb60 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbda9ffb700 1 -- 172.21.15.33:0/892942180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00b860 con 0x7fbdac0adb60 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbda9ffb700 1 -- 172.21.15.33:0/892942180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00bc00 con 0x7fbdac0adb60 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 -- 172.21.15.33:0/892942180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac0adb60 msgr2=0x7fbdac0c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 --2- 172.21.15.33:0/892942180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac0adb60 0x7fbdac0c6820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0099b0 tx=0x7fbd9c009250 comp rx=0 tx=0).stop 2023-10-06T02:46:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 -- 172.21.15.33:0/892942180 shutdown_connections 2023-10-06T02:46:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 --2- 172.21.15.33:0/892942180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac15ace0 0x7fbdac069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 --2- 172.21.15.33:0/892942180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac0adb60 0x7fbdac0c6820 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 -- 172.21.15.33:0/892942180 >> 172.21.15.33:0/892942180 conn(0x7fbdac0c8240 msgr2=0x7fbdac0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 -- 172.21.15.33:0/892942180 shutdown_connections 2023-10-06T02:46:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.045+0000 7fbdabfff700 1 -- 172.21.15.33:0/892942180 wait complete. 2023-10-06T02:46:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 Processor -- start 2023-10-06T02:46:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 -- start start 2023-10-06T02:46:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 0x7fbdac142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac0adb60 0x7fbdac0a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdac15ace0 0x7fbdac0a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0c5530 con 0x7fbdac067970 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0c4920 con 0x7fbdac15ace0 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0c4c00 con 0x7fbdac0adb60 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 0x7fbdac142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 0x7fbdac142340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36038/0 (socket says 172.21.15.33:36038) 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 -- 172.21.15.33:0/171213658 learned_addr learned my addr 172.21.15.33:0/171213658 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdab7fe700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdac15ace0 0x7fbdac0a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaa7fc700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac0adb60 0x7fbdac0a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 -- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac0adb60 msgr2=0x7fbdac0a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac0adb60 0x7fbdac0a44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 -- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdac15ace0 msgr2=0x7fbdac0a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdac15ace0 0x7fbdac0a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 -- 172.21.15.33:0/171213658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0094f0 con 0x7fbdac067970 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaa7fc700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac0adb60 0x7fbdac0a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdaaffd700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 0x7fbdac142340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbdac06cd80 tx=0x7fbd9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.056 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c000d10 con 0x7fbdac067970 2023-10-06T02:46:31.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00bbe0 con 0x7fbdac067970 2023-10-06T02:46:31.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdac0c4ee0 con 0x7fbdac067970 2023-10-06T02:46:31.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c011690 con 0x7fbdac067970 2023-10-06T02:46:31.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.049+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdac142880 con 0x7fbdac067970 2023-10-06T02:46:31.057 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.053+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdac071040 con 0x7fbdac067970 2023-10-06T02:46:31.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.053+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbd9c011a80 con 0x7fbdac067970 2023-10-06T02:46:31.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.053+0000 7fbda2ffd700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbd8c060550 0x7fbd8c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.053+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fbd9c07e320 con 0x7fbdac067970 2023-10-06T02:46:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.053+0000 7fbdaa7fc700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbd8c060550 0x7fbd8c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.057+0000 7fbdaa7fc700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbd8c060550 0x7fbd8c062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbd94007fd0 tx=0x7fbd94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.061+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbd9c04e670 con 0x7fbdac067970 2023-10-06T02:46:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.225+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdac068460 con 0x7fbdac067970 2023-10-06T02:46:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbda2ffd700 1 -- 172.21.15.33:0/171213658 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17071 (secure 0 0 0) 0x7fbd9c051c90 con 0x7fbdac067970 2023-10-06T02:46:31.233 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:31.233 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":44,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:29.309782+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:46:25.764531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T02:46:26.918595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:31.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbd8c060550 msgr2=0x7fbd8c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbd8c060550 0x7fbd8c062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbd94007fd0 tx=0x7fbd94009040 comp rx=0 tx=0).stop 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 msgr2=0x7fbdac142340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 0x7fbdac142340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbdac06cd80 tx=0x7fbd9c017000 comp rx=0 tx=0).stop 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 shutdown_connections 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fbd8c060550 0x7fbd8c062a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbdac0adb60 0x7fbdac0a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbdac15ace0 0x7fbdac0a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 --2- 172.21.15.33:0/171213658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdac067970 0x7fbdac142340 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 >> 172.21.15.33:0/171213658 conn(0x7fbdac0c8240 msgr2=0x7fbdac158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:31.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 shutdown_connections 2023-10-06T02:46:31.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.229+0000 7fbdabfff700 1 -- 172.21.15.33:0/171213658 wait complete. 2023-10-06T02:46:31.248 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T02:46:31.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:46:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 Processor -- start 2023-10-06T02:46:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- start start 2023-10-06T02:46:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e480c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e481613a0 0x7f3e481617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e48067050 con 0x7f3e480680f0 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e480c9a90 con 0x7f3e481613a0 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48067330 con 0x7f3e48160fd0 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e481613a0 0x7f3e481617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4e7e7700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3e480680f0 0x7f3e480c0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60770/0 (socket says 172.21.15.33:60770) 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e481613a0 0x7f3e481617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36044/0 (socket says 172.21.15.33:36044) 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4e7e7700 1 -- 172.21.15.33:0/4201324685 learned_addr learned my addr 172.21.15.33:0/4201324685 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4dfe6700 1 --2- 172.21.15.33:0/4201324685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e480c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 -- 172.21.15.33:0/4201324685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 msgr2=0x7f3e480c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4cfe4700 1 -- 172.21.15.33:0/4201324685 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397102765 0 0) 0x7f3e48067050 con 0x7f3e480680f0 2023-10-06T02:46:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 --2- 172.21.15.33:0/4201324685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e480c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 -- 172.21.15.33:0/4201324685 >> v1:172.21.15.33:6789/0 conn(0x7f3e480680f0 legacy=0x7f3e480c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 -- 172.21.15.33:0/4201324685 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48067610 con 0x7f3e481613a0 2023-10-06T02:46:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4d7e5700 1 --2- 172.21.15.33:0/4201324685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e481613a0 0x7f3e481617f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3e440098e0 tx=0x7f3e440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f99a89443b127ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4cfe4700 1 -- 172.21.15.33:0/4201324685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400b640 con 0x7f3e481613a0 2023-10-06T02:46:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4cfe4700 1 -- 172.21.15.33:0/4201324685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4400b7a0 con 0x7f3e481613a0 2023-10-06T02:46:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e4cfe4700 1 -- 172.21.15.33:0/4201324685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400bb40 con 0x7f3e481613a0 2023-10-06T02:46:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- 172.21.15.33:0/4201324685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e481613a0 msgr2=0x7f3e481617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 --2- 172.21.15.33:0/4201324685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e481613a0 0x7f3e481617f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3e440098e0 tx=0x7f3e440091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- 172.21.15.33:0/4201324685 shutdown_connections 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 --2- 172.21.15.33:0/4201324685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e481613a0 0x7f3e481617f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 --2- 172.21.15.33:0/4201324685 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e480c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- 172.21.15.33:0/4201324685 >> 172.21.15.33:0/4201324685 conn(0x7f3e480c83f0 msgr2=0x7f3e480c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- 172.21.15.33:0/4201324685 shutdown_connections 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.353+0000 7f3e5024a700 1 -- 172.21.15.33:0/4201324685 wait complete. 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 Processor -- start 2023-10-06T02:46:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 -- start start 2023-10-06T02:46:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e480680f0 0x7f3e481480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e48170670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 0x7f3e480af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48171050 con 0x7f3e48170bb0 2023-10-06T02:46:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480aeba0 con 0x7f3e480680f0 2023-10-06T02:46:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4815e2d0 con 0x7f3e48160fd0 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 0x7f3e480af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 0x7f3e480af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36054/0 (socket says 172.21.15.33:36054) 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 -- 172.21.15.33:0/403958038 learned_addr learned my addr 172.21.15.33:0/403958038 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4dfe6700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e480680f0 0x7f3e481480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4d7e5700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e48170670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 -- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 msgr2=0x7f3e48170670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e48170670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 -- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e480680f0 msgr2=0x7f3e481480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e480680f0 0x7f3e481480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 -- 172.21.15.33:0/403958038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44009420 con 0x7f3e48170bb0 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4dfe6700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e480680f0 0x7f3e481480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e4e7e7700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 0x7f3e480af030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000cc90 tx=0x7f3e40001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000cea0 con 0x7f3e48170bb0 2023-10-06T02:46:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000ab70 con 0x7f3e48170bb0 2023-10-06T02:46:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4815e5b0 con 0x7f3e48170bb0 2023-10-06T02:46:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400070f0 con 0x7f3e48170bb0 2023-10-06T02:46:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.357+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4815e890 con 0x7f3e48170bb0 2023-10-06T02:46:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.361+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4815a590 con 0x7f3e48170bb0 2023-10-06T02:46:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.361+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3e40007460 con 0x7f3e48170bb0 2023-10-06T02:46:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.361+0000 7f3e39ffb700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e2c060550 0x7f3e2c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.361+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3e4007cec0 con 0x7f3e48170bb0 2023-10-06T02:46:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.361+0000 7f3e4dfe6700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e2c060550 0x7f3e2c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.361+0000 7f3e4dfe6700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e2c060550 0x7f3e2c062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009d80 tx=0x7f3e3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.373 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.369+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3e4004d290 con 0x7f3e48170bb0 2023-10-06T02:46:31.494 INFO:tasks.ceph.mds.a.smithi033.stderr:2023-10-06T02:46:31.489+0000 7efbf94db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:46:31.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.529+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e480710b0 con 0x7f3e2c060550 2023-10-06T02:46:31.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e39ffb700 1 -- 172.21.15.33:0/403958038 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309598 (secure 0 0 0) 0x7f3e480710b0 con 0x7f3e2c060550 2023-10-06T02:46:31.555 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:31.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e2c060550 msgr2=0x7f3e2c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e2c060550 0x7f3e2c062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009d80 tx=0x7f3e3c008040 comp rx=0 tx=0).stop 2023-10-06T02:46:31.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 msgr2=0x7f3e480af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 0x7f3e480af030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000cc90 tx=0x7f3e40001a80 comp rx=0 tx=0).stop 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 shutdown_connections 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e2c060550 0x7f3e2c062a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e48160fd0 0x7f3e48170670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48170bb0 0x7f3e480af030 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 --2- 172.21.15.33:0/403958038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e480680f0 0x7f3e481480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 >> 172.21.15.33:0/403958038 conn(0x7f3e480c83f0 msgr2=0x7f3e4815ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 shutdown_connections 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.549+0000 7f3e5024a700 1 -- 172.21.15.33:0/403958038 wait complete. 2023-10-06T02:46:31.558 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:46:31.570 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-10-06T02:46:30.582947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383400,"kb_used_data":5096,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591320,"statfs":{"total":773094113280,"available":770653511680,"internally_reserved":0,"allocated":5218304,"data_stored":2946020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.051282"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335955+0000","last_change":"2023-10-06T02:46:28.335955+0000","last_active":"2023-10-06T02:46:28.335955+0000","last_peered":"2023-10-06T02:46:28.335955+0000","last_clean":"2023-10-06T02:46:28.335955+0000","last_became_active":"2023-10-06T02:46:28.335340+0000","last_became_peered":"2023-10-06T02:46:28.335340+0000","last_unstale":"2023-10-06T02:46:28.335955+0000","last_undegraded":"2023-10-06T02:46:28.335955+0000","last_fullsized":"2023-10-06T02:46:28.335955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:36.931513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339465+0000","last_change":"2023-10-06T02:46:28.339465+0000","last_active":"2023-10-06T02:46:28.339465+0000","last_peered":"2023-10-06T02:46:28.339465+0000","last_clean":"2023-10-06T02:46:28.339465+0000","last_became_active":"2023-10-06T02:46:28.339157+0000","last_became_peered":"2023-10-06T02:46:28.339157+0000","last_unstale":"2023-10-06T02:46:28.339465+0000","last_undegraded":"2023-10-06T02:46:28.339465+0000","last_fullsized":"2023-10-06T02:46:28.339465+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:27.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339508+0000","last_change":"2023-10-06T02:46:28.334929+0000","last_active":"2023-10-06T02:46:28.339508+0000","last_peered":"2023-10-06T02:46:28.339508+0000","last_clean":"2023-10-06T02:46:28.339508+0000","last_became_active":"2023-10-06T02:46:28.334653+0000","last_became_peered":"2023-10-06T02:46:28.334653+0000","last_unstale":"2023-10-06T02:46:28.339508+0000","last_undegraded":"2023-10-06T02:46:28.339508+0000","last_fullsized":"2023-10-06T02:46:28.339508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:17:30.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.347217+0000","last_change":"2023-10-06T02:46:28.347217+0000","last_active":"2023-10-06T02:46:28.347217+0000","last_peered":"2023-10-06T02:46:28.347217+0000","last_clean":"2023-10-06T02:46:28.347217+0000","last_became_active":"2023-10-06T02:46:28.344294+0000","last_became_peered":"2023-10-06T02:46:28.344294+0000","last_unstale":"2023-10-06T02:46:28.347217+0000","last_undegraded":"2023-10-06T02:46:28.347217+0000","last_fullsized":"2023-10-06T02:46:28.347217+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:08.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.337687+0000","last_change":"2023-10-06T02:46:28.337687+0000","last_active":"2023-10-06T02:46:28.337687+0000","last_peered":"2023-10-06T02:46:28.337687+0000","last_clean":"2023-10-06T02:46:28.337687+0000","last_became_active":"2023-10-06T02:46:28.337391+0000","last_became_peered":"2023-10-06T02:46:28.337391+0000","last_unstale":"2023-10-06T02:46:28.337687+0000","last_undegraded":"2023-10-06T02:46:28.337687+0000","last_fullsized":"2023-10-06T02:46:28.337687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.111686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.337810+0000","last_change":"2023-10-06T02:46:28.337810+0000","last_active":"2023-10-06T02:46:28.337810+0000","last_peered":"2023-10-06T02:46:28.337810+0000","last_clean":"2023-10-06T02:46:28.337810+0000","last_became_active":"2023-10-06T02:46:28.335855+0000","last_became_peered":"2023-10-06T02:46:28.335855+0000","last_unstale":"2023-10-06T02:46:28.337810+0000","last_undegraded":"2023-10-06T02:46:28.337810+0000","last_fullsized":"2023-10-06T02:46:28.337810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:34.618192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335300+0000","last_change":"2023-10-06T02:46:28.335300+0000","last_active":"2023-10-06T02:46:28.335300+0000","last_peered":"2023-10-06T02:46:28.335300+0000","last_clean":"2023-10-06T02:46:28.335300+0000","last_became_active":"2023-10-06T02:46:28.334861+0000","last_became_peered":"2023-10-06T02:46:28.334861+0000","last_unstale":"2023-10-06T02:46:28.335300+0000","last_undegraded":"2023-10-06T02:46:28.335300+0000","last_fullsized":"2023-10-06T02:46:28.335300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:21.922252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344494+0000","last_change":"2023-10-06T02:46:28.344494+0000","last_active":"2023-10-06T02:46:28.344494+0000","last_peered":"2023-10-06T02:46:28.344494+0000","last_clean":"2023-10-06T02:46:28.344494+0000","last_became_active":"2023-10-06T02:46:28.341901+0000","last_became_peered":"2023-10-06T02:46:28.341901+0000","last_unstale":"2023-10-06T02:46:28.344494+0000","last_undegraded":"2023-10-06T02:46:28.344494+0000","last_fullsized":"2023-10-06T02:46:28.344494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:16.888525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.348058+0000","last_change":"2023-10-06T02:46:28.348058+0000","last_active":"2023-10-06T02:46:28.348058+0000","last_peered":"2023-10-06T02:46:28.348058+0000","last_clean":"2023-10-06T02:46:28.348058+0000","last_became_active":"2023-10-06T02:46:28.347744+0000","last_became_peered":"2023-10-06T02:46:28.347744+0000","last_unstale":"2023-10-06T02:46:28.348058+0000","last_undegraded":"2023-10-06T02:46:28.348058+0000","last_fullsized":"2023-10-06T02:46:28.348058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:06.262037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341686+0000","last_change":"2023-10-06T02:46:28.341686+0000","last_active":"2023-10-06T02:46:28.341686+0000","last_peered":"2023-10-06T02:46:28.341686+0000","last_clean":"2023-10-06T02:46:28.341686+0000","last_became_active":"2023-10-06T02:46:28.340990+0000","last_became_peered":"2023-10-06T02:46:28.340990+0000","last_unstale":"2023-10-06T02:46:28.341686+0000","last_undegraded":"2023-10-06T02:46:28.341686+0000","last_fullsized":"2023-10-06T02:46:28.341686+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:16.514623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344398+0000","last_change":"2023-10-06T02:46:28.344398+0000","last_active":"2023-10-06T02:46:28.344398+0000","last_peered":"2023-10-06T02:46:28.344398+0000","last_clean":"2023-10-06T02:46:28.344398+0000","last_became_active":"2023-10-06T02:46:28.344068+0000","last_became_peered":"2023-10-06T02:46:28.344068+0000","last_unstale":"2023-10-06T02:46:28.344398+0000","last_undegraded":"2023-10-06T02:46:28.344398+0000","last_fullsized":"2023-10-06T02:46:28.344398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:04.203288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.337245+0000","last_change":"2023-10-06T02:46:28.337245+0000","last_active":"2023-10-06T02:46:28.337245+0000","last_peered":"2023-10-06T02:46:28.337245+0000","last_clean":"2023-10-06T02:46:28.337245+0000","last_became_active":"2023-10-06T02:46:28.336773+0000","last_became_peered":"2023-10-06T02:46:28.336773+0000","last_unstale":"2023-10-06T02:46:28.337245+0000","last_undegraded":"2023-10-06T02:46:28.337245+0000","last_fullsized":"2023-10-06T02:46:28.337245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:16.514623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.345791+0000","last_change":"2023-10-06T02:46:28.338959+0000","last_active":"2023-10-06T02:46:28.345791+0000","last_peered":"2023-10-06T02:46:28.345791+0000","last_clean":"2023-10-06T02:46:28.345791+0000","last_became_active":"2023-10-06T02:46:28.338610+0000","last_became_peered":"2023-10-06T02:46:28.338610+0000","last_unstale":"2023-10-06T02:46:28.345791+0000","last_undegraded":"2023-10-06T02:46:28.345791+0000","last_fullsized":"2023-10-06T02:46:28.345791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:18.705652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344964+0000","last_change":"2023-10-06T02:46:28.344964+0000","last_active":"2023-10-06T02:46:28.344964+0000","last_peered":"2023-10-06T02:46:28.344964+0000","last_clean":"2023-10-06T02:46:28.344964+0000","last_became_active":"2023-10-06T02:46:28.344116+0000","last_became_peered":"2023-10-06T02:46:28.344116+0000","last_unstale":"2023-10-06T02:46:28.344964+0000","last_undegraded":"2023-10-06T02:46:28.344964+0000","last_fullsized":"2023-10-06T02:46:28.344964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:26.287253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341259+0000","last_change":"2023-10-06T02:46:28.341259+0000","last_active":"2023-10-06T02:46:28.341259+0000","last_peered":"2023-10-06T02:46:28.341259+0000","last_clean":"2023-10-06T02:46:28.341259+0000","last_became_active":"2023-10-06T02:46:28.340989+0000","last_became_peered":"2023-10-06T02:46:28.340989+0000","last_unstale":"2023-10-06T02:46:28.341259+0000","last_undegraded":"2023-10-06T02:46:28.341259+0000","last_fullsized":"2023-10-06T02:46:28.341259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:27.498868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.346702+0000","last_change":"2023-10-06T02:46:28.344150+0000","last_active":"2023-10-06T02:46:28.346702+0000","last_peered":"2023-10-06T02:46:28.346702+0000","last_clean":"2023-10-06T02:46:28.346702+0000","last_became_active":"2023-10-06T02:46:28.343816+0000","last_became_peered":"2023-10-06T02:46:28.343816+0000","last_unstale":"2023-10-06T02:46:28.346702+0000","last_undegraded":"2023-10-06T02:46:28.346702+0000","last_fullsized":"2023-10-06T02:46:28.346702+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:58.728344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.330309+0000","last_change":"2023-10-06T02:46:28.328175+0000","last_active":"2023-10-06T02:46:28.330309+0000","last_peered":"2023-10-06T02:46:28.330309+0000","last_clean":"2023-10-06T02:46:28.330309+0000","last_became_active":"2023-10-06T02:46:28.327825+0000","last_became_peered":"2023-10-06T02:46:28.327825+0000","last_unstale":"2023-10-06T02:46:28.330309+0000","last_undegraded":"2023-10-06T02:46:28.330309+0000","last_fullsized":"2023-10-06T02:46:28.330309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:47.028557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342417+0000","last_change":"2023-10-06T02:46:28.338351+0000","last_active":"2023-10-06T02:46:28.342417+0000","last_peered":"2023-10-06T02:46:28.342417+0000","last_clean":"2023-10-06T02:46:28.342417+0000","last_became_active":"2023-10-06T02:46:28.337909+0000","last_became_peered":"2023-10-06T02:46:28.337909+0000","last_unstale":"2023-10-06T02:46:28.342417+0000","last_undegraded":"2023-10-06T02:46:28.342417+0000","last_fullsized":"2023-10-06T02:46:28.342417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:46.909260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342832+0000","last_change":"2023-10-06T02:46:28.337692+0000","last_active":"2023-10-06T02:46:28.342832+0000","last_peered":"2023-10-06T02:46:28.342832+0000","last_clean":"2023-10-06T02:46:28.342832+0000","last_became_active":"2023-10-06T02:46:28.337393+0000","last_became_peered":"2023-10-06T02:46:28.337393+0000","last_unstale":"2023-10-06T02:46:28.342832+0000","last_undegraded":"2023-10-06T02:46:28.342832+0000","last_fullsized":"2023-10-06T02:46:28.342832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:10.904244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342088+0000","last_change":"2023-10-06T02:46:28.337814+0000","last_active":"2023-10-06T02:46:28.342088+0000","last_peered":"2023-10-06T02:46:28.342088+0000","last_clean":"2023-10-06T02:46:28.342088+0000","last_became_active":"2023-10-06T02:46:28.337511+0000","last_became_peered":"2023-10-06T02:46:28.337511+0000","last_unstale":"2023-10-06T02:46:28.342088+0000","last_undegraded":"2023-10-06T02:46:28.342088+0000","last_fullsized":"2023-10-06T02:46:28.342088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:07.273611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342495+0000","last_change":"2023-10-06T02:46:28.342495+0000","last_active":"2023-10-06T02:46:28.342495+0000","last_peered":"2023-10-06T02:46:28.342495+0000","last_clean":"2023-10-06T02:46:28.342495+0000","last_became_active":"2023-10-06T02:46:28.342106+0000","last_became_peered":"2023-10-06T02:46:28.342106+0000","last_unstale":"2023-10-06T02:46:28.342495+0000","last_undegraded":"2023-10-06T02:46:28.342495+0000","last_fullsized":"2023-10-06T02:46:28.342495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:07.886293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341380+0000","last_change":"2023-10-06T02:46:28.341380+0000","last_active":"2023-10-06T02:46:28.341380+0000","last_peered":"2023-10-06T02:46:28.341380+0000","last_clean":"2023-10-06T02:46:28.341380+0000","last_became_active":"2023-10-06T02:46:28.340902+0000","last_became_peered":"2023-10-06T02:46:28.340902+0000","last_unstale":"2023-10-06T02:46:28.341380+0000","last_undegraded":"2023-10-06T02:46:28.341380+0000","last_fullsized":"2023-10-06T02:46:28.341380+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:43.589676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342726+0000","last_change":"2023-10-06T02:46:28.336390+0000","last_active":"2023-10-06T02:46:28.342726+0000","last_peered":"2023-10-06T02:46:28.342726+0000","last_clean":"2023-10-06T02:46:28.342726+0000","last_became_active":"2023-10-06T02:46:28.335889+0000","last_became_peered":"2023-10-06T02:46:28.335889+0000","last_unstale":"2023-10-06T02:46:28.342726+0000","last_undegraded":"2023-10-06T02:46:28.342726+0000","last_fullsized":"2023-10-06T02:46:28.342726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:07.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342769+0000","last_change":"2023-10-06T02:46:28.337178+0000","last_active":"2023-10-06T02:46:28.342769+0000","last_peered":"2023-10-06T02:46:28.342769+0000","last_clean":"2023-10-06T02:46:28.342769+0000","last_became_active":"2023-10-06T02:46:28.336876+0000","last_became_peered":"2023-10-06T02:46:28.336876+0000","last_unstale":"2023-10-06T02:46:28.342769+0000","last_undegraded":"2023-10-06T02:46:28.342769+0000","last_fullsized":"2023-10-06T02:46:28.342769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.678648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342563+0000","last_change":"2023-10-06T02:46:28.332897+0000","last_active":"2023-10-06T02:46:28.342563+0000","last_peered":"2023-10-06T02:46:28.342563+0000","last_clean":"2023-10-06T02:46:28.342563+0000","last_became_active":"2023-10-06T02:46:28.332573+0000","last_became_peered":"2023-10-06T02:46:28.332573+0000","last_unstale":"2023-10-06T02:46:28.342563+0000","last_undegraded":"2023-10-06T02:46:28.342563+0000","last_fullsized":"2023-10-06T02:46:28.342563+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:10.254146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.346118+0000","last_change":"2023-10-06T02:46:28.339816+0000","last_active":"2023-10-06T02:46:28.346118+0000","last_peered":"2023-10-06T02:46:28.346118+0000","last_clean":"2023-10-06T02:46:28.346118+0000","last_became_active":"2023-10-06T02:46:28.338734+0000","last_became_peered":"2023-10-06T02:46:28.338734+0000","last_unstale":"2023-10-06T02:46:28.346118+0000","last_undegraded":"2023-10-06T02:46:28.346118+0000","last_fullsized":"2023-10-06T02:46:28.346118+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:03.223296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.330460+0000","last_change":"2023-10-06T02:46:28.329055+0000","last_active":"2023-10-06T02:46:28.330460+0000","last_peered":"2023-10-06T02:46:28.330460+0000","last_clean":"2023-10-06T02:46:28.330460+0000","last_became_active":"2023-10-06T02:46:28.328774+0000","last_became_peered":"2023-10-06T02:46:28.328774+0000","last_unstale":"2023-10-06T02:46:28.330460+0000","last_undegraded":"2023-10-06T02:46:28.330460+0000","last_fullsized":"2023-10-06T02:46:28.330460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:10.157128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334636+0000","last_change":"2023-10-06T02:46:28.334636+0000","last_active":"2023-10-06T02:46:28.334636+0000","last_peered":"2023-10-06T02:46:28.334636+0000","last_clean":"2023-10-06T02:46:28.334636+0000","last_became_active":"2023-10-06T02:46:28.334265+0000","last_became_peered":"2023-10-06T02:46:28.334265+0000","last_unstale":"2023-10-06T02:46:28.334636+0000","last_undegraded":"2023-10-06T02:46:28.334636+0000","last_fullsized":"2023-10-06T02:46:28.334636+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:43.589676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342579+0000","last_change":"2023-10-06T02:46:28.333000+0000","last_active":"2023-10-06T02:46:28.342579+0000","last_peered":"2023-10-06T02:46:28.342579+0000","last_clean":"2023-10-06T02:46:28.342579+0000","last_became_active":"2023-10-06T02:46:28.332673+0000","last_became_peered":"2023-10-06T02:46:28.332673+0000","last_unstale":"2023-10-06T02:46:28.342579+0000","last_undegraded":"2023-10-06T02:46:28.342579+0000","last_fullsized":"2023-10-06T02:46:28.342579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:30.592850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339600+0000","last_change":"2023-10-06T02:46:28.335854+0000","last_active":"2023-10-06T02:46:28.339600+0000","last_peered":"2023-10-06T02:46:28.339600+0000","last_clean":"2023-10-06T02:46:28.339600+0000","last_became_active":"2023-10-06T02:46:28.335565+0000","last_became_peered":"2023-10-06T02:46:28.335565+0000","last_unstale":"2023-10-06T02:46:28.339600+0000","last_undegraded":"2023-10-06T02:46:28.339600+0000","last_fullsized":"2023-10-06T02:46:28.339600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:07.794047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.338585+0000","last_change":"2023-10-06T02:46:28.338585+0000","last_active":"2023-10-06T02:46:28.338585+0000","last_peered":"2023-10-06T02:46:28.338585+0000","last_clean":"2023-10-06T02:46:28.338585+0000","last_became_active":"2023-10-06T02:46:28.335815+0000","last_became_peered":"2023-10-06T02:46:28.335815+0000","last_unstale":"2023-10-06T02:46:28.338585+0000","last_undegraded":"2023-10-06T02:46:28.338585+0000","last_fullsized":"2023-10-06T02:46:28.338585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:44.775700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.346319+0000","last_change":"2023-10-06T02:46:28.343557+0000","last_active":"2023-10-06T02:46:28.346319+0000","last_peered":"2023-10-06T02:46:28.346319+0000","last_clean":"2023-10-06T02:46:28.346319+0000","last_became_active":"2023-10-06T02:46:28.343239+0000","last_became_peered":"2023-10-06T02:46:28.343239+0000","last_unstale":"2023-10-06T02:46:28.346319+0000","last_undegraded":"2023-10-06T02:46:28.346319+0000","last_fullsized":"2023-10-06T02:46:28.346319+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:06.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335227+0000","last_change":"2023-10-06T02:46:28.335227+0000","last_active":"2023-10-06T02:46:28.335227+0000","last_peered":"2023-10-06T02:46:28.335227+0000","last_clean":"2023-10-06T02:46:28.335227+0000","last_became_active":"2023-10-06T02:46:28.334935+0000","last_became_peered":"2023-10-06T02:46:28.334935+0000","last_unstale":"2023-10-06T02:46:28.335227+0000","last_undegraded":"2023-10-06T02:46:28.335227+0000","last_fullsized":"2023-10-06T02:46:28.335227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:05.138628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.573580+0000","last_change":"2023-10-06T02:46:27.557700+0000","last_active":"2023-10-06T02:46:28.573580+0000","last_peered":"2023-10-06T02:46:28.573580+0000","last_clean":"2023-10-06T02:46:28.573580+0000","last_became_active":"2023-10-06T02:46:27.557396+0000","last_became_peered":"2023-10-06T02:46:27.557396+0000","last_unstale":"2023-10-06T02:46:28.573580+0000","last_undegraded":"2023-10-06T02:46:28.573580+0000","last_fullsized":"2023-10-06T02:46:28.573580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:15.146702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.340191+0000","last_change":"2023-10-06T02:46:28.329957+0000","last_active":"2023-10-06T02:46:28.340191+0000","last_peered":"2023-10-06T02:46:28.340191+0000","last_clean":"2023-10-06T02:46:28.340191+0000","last_became_active":"2023-10-06T02:46:28.329675+0000","last_became_peered":"2023-10-06T02:46:28.329675+0000","last_unstale":"2023-10-06T02:46:28.340191+0000","last_undegraded":"2023-10-06T02:46:28.340191+0000","last_fullsized":"2023-10-06T02:46:28.340191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:04.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572595+0000","last_change":"2023-10-06T02:46:27.561770+0000","last_active":"2023-10-06T02:46:28.572595+0000","last_peered":"2023-10-06T02:46:28.572595+0000","last_clean":"2023-10-06T02:46:28.572595+0000","last_became_active":"2023-10-06T02:46:27.561473+0000","last_became_peered":"2023-10-06T02:46:27.561473+0000","last_unstale":"2023-10-06T02:46:28.572595+0000","last_undegraded":"2023-10-06T02:46:28.572595+0000","last_fullsized":"2023-10-06T02:46:28.572595+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:35.919063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.338447+0000","last_change":"2023-10-06T02:46:28.338447+0000","last_active":"2023-10-06T02:46:28.338447+0000","last_peered":"2023-10-06T02:46:28.338447+0000","last_clean":"2023-10-06T02:46:28.338447+0000","last_became_active":"2023-10-06T02:46:28.338137+0000","last_became_peered":"2023-10-06T02:46:28.338137+0000","last_unstale":"2023-10-06T02:46:28.338447+0000","last_undegraded":"2023-10-06T02:46:28.338447+0000","last_fullsized":"2023-10-06T02:46:28.338447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:54.979203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.570579+0000","last_change":"2023-10-06T02:46:27.568255+0000","last_active":"2023-10-06T02:46:28.570579+0000","last_peered":"2023-10-06T02:46:28.570579+0000","last_clean":"2023-10-06T02:46:28.570579+0000","last_became_active":"2023-10-06T02:46:27.567918+0000","last_became_peered":"2023-10-06T02:46:27.567918+0000","last_unstale":"2023-10-06T02:46:28.570579+0000","last_undegraded":"2023-10-06T02:46:28.570579+0000","last_fullsized":"2023-10-06T02:46:28.570579+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:05.364568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.331925+0000","last_change":"2023-10-06T02:46:28.331925+0000","last_active":"2023-10-06T02:46:28.331925+0000","last_peered":"2023-10-06T02:46:28.331925+0000","last_clean":"2023-10-06T02:46:28.331925+0000","last_became_active":"2023-10-06T02:46:28.331622+0000","last_became_peered":"2023-10-06T02:46:28.331622+0000","last_unstale":"2023-10-06T02:46:28.331925+0000","last_undegraded":"2023-10-06T02:46:28.331925+0000","last_fullsized":"2023-10-06T02:46:28.331925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:39.306831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.571503+0000","last_change":"2023-10-06T02:46:27.565565+0000","last_active":"2023-10-06T02:46:28.571503+0000","last_peered":"2023-10-06T02:46:28.571503+0000","last_clean":"2023-10-06T02:46:28.571503+0000","last_became_active":"2023-10-06T02:46:27.565189+0000","last_became_peered":"2023-10-06T02:46:27.565189+0000","last_unstale":"2023-10-06T02:46:28.571503+0000","last_undegraded":"2023-10-06T02:46:28.571503+0000","last_fullsized":"2023-10-06T02:46:28.571503+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:49.647098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.330369+0000","last_change":"2023-10-06T02:46:28.328357+0000","last_active":"2023-10-06T02:46:28.330369+0000","last_peered":"2023-10-06T02:46:28.330369+0000","last_clean":"2023-10-06T02:46:28.330369+0000","last_became_active":"2023-10-06T02:46:28.328001+0000","last_became_peered":"2023-10-06T02:46:28.328001+0000","last_unstale":"2023-10-06T02:46:28.330369+0000","last_undegraded":"2023-10-06T02:46:28.330369+0000","last_fullsized":"2023-10-06T02:46:28.330369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:59.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.327552+0000","last_change":"2023-10-06T02:46:27.570497+0000","last_active":"2023-10-06T02:46:28.327552+0000","last_peered":"2023-10-06T02:46:28.327552+0000","last_clean":"2023-10-06T02:46:28.327552+0000","last_became_active":"2023-10-06T02:46:27.567022+0000","last_became_peered":"2023-10-06T02:46:27.567022+0000","last_unstale":"2023-10-06T02:46:28.327552+0000","last_undegraded":"2023-10-06T02:46:28.327552+0000","last_fullsized":"2023-10-06T02:46:28.327552+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:29.323681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335169+0000","last_change":"2023-10-06T02:46:28.335169+0000","last_active":"2023-10-06T02:46:28.335169+0000","last_peered":"2023-10-06T02:46:28.335169+0000","last_clean":"2023-10-06T02:46:28.335169+0000","last_became_active":"2023-10-06T02:46:28.334613+0000","last_became_peered":"2023-10-06T02:46:28.334613+0000","last_unstale":"2023-10-06T02:46:28.335169+0000","last_undegraded":"2023-10-06T02:46:28.335169+0000","last_fullsized":"2023-10-06T02:46:28.335169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:46.909260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.570456+0000","last_change":"2023-10-06T02:46:27.567470+0000","last_active":"2023-10-06T02:46:28.570456+0000","last_peered":"2023-10-06T02:46:28.570456+0000","last_clean":"2023-10-06T02:46:28.570456+0000","last_became_active":"2023-10-06T02:46:27.567183+0000","last_became_peered":"2023-10-06T02:46:27.567183+0000","last_unstale":"2023-10-06T02:46:28.570456+0000","last_undegraded":"2023-10-06T02:46:28.570456+0000","last_fullsized":"2023-10-06T02:46:28.570456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:20.909802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341494+0000","last_change":"2023-10-06T02:46:28.341494+0000","last_active":"2023-10-06T02:46:28.341494+0000","last_peered":"2023-10-06T02:46:28.341494+0000","last_clean":"2023-10-06T02:46:28.341494+0000","last_became_active":"2023-10-06T02:46:28.340853+0000","last_became_peered":"2023-10-06T02:46:28.340853+0000","last_unstale":"2023-10-06T02:46:28.341494+0000","last_undegraded":"2023-10-06T02:46:28.341494+0000","last_fullsized":"2023-10-06T02:46:28.341494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.111686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.570267+0000","last_change":"2023-10-06T02:46:27.567473+0000","last_active":"2023-10-06T02:46:27.570267+0000","last_peered":"2023-10-06T02:46:27.570267+0000","last_clean":"2023-10-06T02:46:27.570267+0000","last_became_active":"2023-10-06T02:46:27.567163+0000","last_became_peered":"2023-10-06T02:46:27.567163+0000","last_unstale":"2023-10-06T02:46:27.570267+0000","last_undegraded":"2023-10-06T02:46:27.570267+0000","last_fullsized":"2023-10-06T02:46:27.570267+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:56.478162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342707+0000","last_change":"2023-10-06T02:46:28.336189+0000","last_active":"2023-10-06T02:46:28.342707+0000","last_peered":"2023-10-06T02:46:28.342707+0000","last_clean":"2023-10-06T02:46:28.342707+0000","last_became_active":"2023-10-06T02:46:28.335487+0000","last_became_peered":"2023-10-06T02:46:28.335487+0000","last_unstale":"2023-10-06T02:46:28.342707+0000","last_undegraded":"2023-10-06T02:46:28.342707+0000","last_fullsized":"2023-10-06T02:46:28.342707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:45.410932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.566975+0000","last_change":"2023-10-06T02:46:27.564075+0000","last_active":"2023-10-06T02:46:27.566975+0000","last_peered":"2023-10-06T02:46:27.566975+0000","last_clean":"2023-10-06T02:46:27.566975+0000","last_became_active":"2023-10-06T02:46:27.563766+0000","last_became_peered":"2023-10-06T02:46:27.563766+0000","last_unstale":"2023-10-06T02:46:27.566975+0000","last_undegraded":"2023-10-06T02:46:27.566975+0000","last_fullsized":"2023-10-06T02:46:27.566975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:35.287886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342397+0000","last_change":"2023-10-06T02:46:28.338161+0000","last_active":"2023-10-06T02:46:28.342397+0000","last_peered":"2023-10-06T02:46:28.342397+0000","last_clean":"2023-10-06T02:46:28.342397+0000","last_became_active":"2023-10-06T02:46:28.337552+0000","last_became_peered":"2023-10-06T02:46:28.337552+0000","last_unstale":"2023-10-06T02:46:28.342397+0000","last_undegraded":"2023-10-06T02:46:28.342397+0000","last_fullsized":"2023-10-06T02:46:28.342397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:04.203288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.320395+0000","last_change":"2023-10-06T02:46:27.566209+0000","last_active":"2023-10-06T02:46:28.320395+0000","last_peered":"2023-10-06T02:46:28.320395+0000","last_clean":"2023-10-06T02:46:28.320395+0000","last_became_active":"2023-10-06T02:46:27.565930+0000","last_became_peered":"2023-10-06T02:46:27.565930+0000","last_unstale":"2023-10-06T02:46:28.320395+0000","last_undegraded":"2023-10-06T02:46:28.320395+0000","last_fullsized":"2023-10-06T02:46:28.320395+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:25.274803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.336186+0000","last_change":"2023-10-06T02:46:28.336186+0000","last_active":"2023-10-06T02:46:28.336186+0000","last_peered":"2023-10-06T02:46:28.336186+0000","last_clean":"2023-10-06T02:46:28.336186+0000","last_became_active":"2023-10-06T02:46:28.335669+0000","last_became_peered":"2023-10-06T02:46:28.335669+0000","last_unstale":"2023-10-06T02:46:28.336186+0000","last_undegraded":"2023-10-06T02:46:28.336186+0000","last_fullsized":"2023-10-06T02:46:28.336186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:06.262037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574550+0000","last_change":"2023-10-06T02:46:27.568986+0000","last_active":"2023-10-06T02:46:27.574550+0000","last_peered":"2023-10-06T02:46:27.574550+0000","last_clean":"2023-10-06T02:46:27.574550+0000","last_became_active":"2023-10-06T02:46:27.568702+0000","last_became_peered":"2023-10-06T02:46:27.568702+0000","last_unstale":"2023-10-06T02:46:27.574550+0000","last_undegraded":"2023-10-06T02:46:27.574550+0000","last_fullsized":"2023-10-06T02:46:27.574550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:35.431730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339031+0000","last_change":"2023-10-06T02:46:28.339031+0000","last_active":"2023-10-06T02:46:28.339031+0000","last_peered":"2023-10-06T02:46:28.339031+0000","last_clean":"2023-10-06T02:46:28.339031+0000","last_became_active":"2023-10-06T02:46:28.338375+0000","last_became_peered":"2023-10-06T02:46:28.338375+0000","last_unstale":"2023-10-06T02:46:28.339031+0000","last_undegraded":"2023-10-06T02:46:28.339031+0000","last_fullsized":"2023-10-06T02:46:28.339031+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:25.554675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572741+0000","last_change":"2023-10-06T02:46:27.568485+0000","last_active":"2023-10-06T02:46:28.572741+0000","last_peered":"2023-10-06T02:46:28.572741+0000","last_clean":"2023-10-06T02:46:28.572741+0000","last_became_active":"2023-10-06T02:46:27.567854+0000","last_became_peered":"2023-10-06T02:46:27.567854+0000","last_unstale":"2023-10-06T02:46:28.572741+0000","last_undegraded":"2023-10-06T02:46:28.572741+0000","last_fullsized":"2023-10-06T02:46:28.572741+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:00.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334936+0000","last_change":"2023-10-06T02:46:28.334936+0000","last_active":"2023-10-06T02:46:28.334936+0000","last_peered":"2023-10-06T02:46:28.334936+0000","last_clean":"2023-10-06T02:46:28.334936+0000","last_became_active":"2023-10-06T02:46:28.334647+0000","last_became_peered":"2023-10-06T02:46:28.334647+0000","last_unstale":"2023-10-06T02:46:28.334936+0000","last_undegraded":"2023-10-06T02:46:28.334936+0000","last_fullsized":"2023-10-06T02:46:28.334936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:26.287253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.578034+0000","last_change":"2023-10-06T02:46:27.562903+0000","last_active":"2023-10-06T02:46:28.578034+0000","last_peered":"2023-10-06T02:46:28.578034+0000","last_clean":"2023-10-06T02:46:28.578034+0000","last_became_active":"2023-10-06T02:46:27.562623+0000","last_became_peered":"2023-10-06T02:46:27.562623+0000","last_unstale":"2023-10-06T02:46:28.578034+0000","last_undegraded":"2023-10-06T02:46:28.578034+0000","last_fullsized":"2023-10-06T02:46:28.578034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:05.633789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342860+0000","last_change":"2023-10-06T02:46:28.337897+0000","last_active":"2023-10-06T02:46:28.342860+0000","last_peered":"2023-10-06T02:46:28.342860+0000","last_clean":"2023-10-06T02:46:28.342860+0000","last_became_active":"2023-10-06T02:46:28.337513+0000","last_became_peered":"2023-10-06T02:46:28.337513+0000","last_unstale":"2023-10-06T02:46:28.342860+0000","last_undegraded":"2023-10-06T02:46:28.342860+0000","last_fullsized":"2023-10-06T02:46:28.342860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.402447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572555+0000","last_change":"2023-10-06T02:46:27.572135+0000","last_active":"2023-10-06T02:46:28.572555+0000","last_peered":"2023-10-06T02:46:28.572555+0000","last_clean":"2023-10-06T02:46:28.572555+0000","last_became_active":"2023-10-06T02:46:27.570782+0000","last_became_peered":"2023-10-06T02:46:27.570782+0000","last_unstale":"2023-10-06T02:46:28.572555+0000","last_undegraded":"2023-10-06T02:46:28.572555+0000","last_fullsized":"2023-10-06T02:46:28.572555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:43.087819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.336744+0000","last_change":"2023-10-06T02:46:28.336744+0000","last_active":"2023-10-06T02:46:28.336744+0000","last_peered":"2023-10-06T02:46:28.336744+0000","last_clean":"2023-10-06T02:46:28.336744+0000","last_became_active":"2023-10-06T02:46:28.336439+0000","last_became_peered":"2023-10-06T02:46:28.336439+0000","last_unstale":"2023-10-06T02:46:28.336744+0000","last_undegraded":"2023-10-06T02:46:28.336744+0000","last_fullsized":"2023-10-06T02:46:28.336744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:07.794047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572402+0000","last_change":"2023-10-06T02:46:27.566874+0000","last_active":"2023-10-06T02:46:27.572402+0000","last_peered":"2023-10-06T02:46:27.572402+0000","last_clean":"2023-10-06T02:46:27.572402+0000","last_became_active":"2023-10-06T02:46:27.566587+0000","last_became_peered":"2023-10-06T02:46:27.566587+0000","last_unstale":"2023-10-06T02:46:27.572402+0000","last_undegraded":"2023-10-06T02:46:27.572402+0000","last_fullsized":"2023-10-06T02:46:27.572402+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:04.126178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335226+0000","last_change":"2023-10-06T02:46:28.335226+0000","last_active":"2023-10-06T02:46:28.335226+0000","last_peered":"2023-10-06T02:46:28.335226+0000","last_clean":"2023-10-06T02:46:28.335226+0000","last_became_active":"2023-10-06T02:46:28.334935+0000","last_became_peered":"2023-10-06T02:46:28.334935+0000","last_unstale":"2023-10-06T02:46:28.335226+0000","last_undegraded":"2023-10-06T02:46:28.335226+0000","last_fullsized":"2023-10-06T02:46:28.335226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:54.738602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.581409+0000","last_change":"2023-10-06T02:46:27.571174+0000","last_active":"2023-10-06T02:46:27.581409+0000","last_peered":"2023-10-06T02:46:27.581409+0000","last_clean":"2023-10-06T02:46:27.581409+0000","last_became_active":"2023-10-06T02:46:27.570370+0000","last_became_peered":"2023-10-06T02:46:27.570370+0000","last_unstale":"2023-10-06T02:46:27.581409+0000","last_undegraded":"2023-10-06T02:46:27.581409+0000","last_fullsized":"2023-10-06T02:46:27.581409+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:16.553122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.340871+0000","last_change":"2023-10-06T02:46:28.340871+0000","last_active":"2023-10-06T02:46:28.340871+0000","last_peered":"2023-10-06T02:46:28.340871+0000","last_clean":"2023-10-06T02:46:28.340871+0000","last_became_active":"2023-10-06T02:46:28.340360+0000","last_became_peered":"2023-10-06T02:46:28.340360+0000","last_unstale":"2023-10-06T02:46:28.340871+0000","last_undegraded":"2023-10-06T02:46:28.340871+0000","last_fullsized":"2023-10-06T02:46:28.340871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:08.877596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.577845+0000","last_change":"2023-10-06T02:46:27.568740+0000","last_active":"2023-10-06T02:46:28.577845+0000","last_peered":"2023-10-06T02:46:28.577845+0000","last_clean":"2023-10-06T02:46:28.577845+0000","last_became_active":"2023-10-06T02:46:27.568398+0000","last_became_peered":"2023-10-06T02:46:27.568398+0000","last_unstale":"2023-10-06T02:46:28.577845+0000","last_undegraded":"2023-10-06T02:46:28.577845+0000","last_fullsized":"2023-10-06T02:46:28.577845+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:05.364568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335798+0000","last_change":"2023-10-06T02:46:28.331749+0000","last_active":"2023-10-06T02:46:28.335798+0000","last_peered":"2023-10-06T02:46:28.335798+0000","last_clean":"2023-10-06T02:46:28.335798+0000","last_became_active":"2023-10-06T02:46:28.331456+0000","last_became_peered":"2023-10-06T02:46:28.331456+0000","last_unstale":"2023-10-06T02:46:28.335798+0000","last_undegraded":"2023-10-06T02:46:28.335798+0000","last_fullsized":"2023-10-06T02:46:28.335798+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:46:56.307796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.573118+0000","last_change":"2023-10-06T02:46:27.571276+0000","last_active":"2023-10-06T02:46:28.573118+0000","last_peered":"2023-10-06T02:46:28.573118+0000","last_clean":"2023-10-06T02:46:28.573118+0000","last_became_active":"2023-10-06T02:46:27.570942+0000","last_became_peered":"2023-10-06T02:46:27.570942+0000","last_unstale":"2023-10-06T02:46:28.573118+0000","last_undegraded":"2023-10-06T02:46:28.573118+0000","last_fullsized":"2023-10-06T02:46:28.573118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:36.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.336910+0000","last_change":"2023-10-06T02:46:28.336910+0000","last_active":"2023-10-06T02:46:28.336910+0000","last_peered":"2023-10-06T02:46:28.336910+0000","last_clean":"2023-10-06T02:46:28.336910+0000","last_became_active":"2023-10-06T02:46:28.336570+0000","last_became_peered":"2023-10-06T02:46:28.336570+0000","last_unstale":"2023-10-06T02:46:28.336910+0000","last_undegraded":"2023-10-06T02:46:28.336910+0000","last_fullsized":"2023-10-06T02:46:28.336910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:23.332330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572813+0000","last_change":"2023-10-06T02:46:27.562648+0000","last_active":"2023-10-06T02:46:28.572813+0000","last_peered":"2023-10-06T02:46:28.572813+0000","last_clean":"2023-10-06T02:46:28.572813+0000","last_became_active":"2023-10-06T02:46:27.562431+0000","last_became_peered":"2023-10-06T02:46:27.562431+0000","last_unstale":"2023-10-06T02:46:28.572813+0000","last_undegraded":"2023-10-06T02:46:28.572813+0000","last_fullsized":"2023-10-06T02:46:28.572813+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:07.865146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342663+0000","last_change":"2023-10-06T02:46:28.334316+0000","last_active":"2023-10-06T02:46:28.342663+0000","last_peered":"2023-10-06T02:46:28.342663+0000","last_clean":"2023-10-06T02:46:28.342663+0000","last_became_active":"2023-10-06T02:46:28.334098+0000","last_became_peered":"2023-10-06T02:46:28.334098+0000","last_unstale":"2023-10-06T02:46:28.342663+0000","last_undegraded":"2023-10-06T02:46:28.342663+0000","last_fullsized":"2023-10-06T02:46:28.342663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:10.268820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.325555+0000","last_change":"2023-10-06T02:46:27.567563+0000","last_active":"2023-10-06T02:46:28.325555+0000","last_peered":"2023-10-06T02:46:28.325555+0000","last_clean":"2023-10-06T02:46:28.325555+0000","last_became_active":"2023-10-06T02:46:27.567238+0000","last_became_peered":"2023-10-06T02:46:27.567238+0000","last_unstale":"2023-10-06T02:46:28.325555+0000","last_undegraded":"2023-10-06T02:46:28.325555+0000","last_fullsized":"2023-10-06T02:46:28.325555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:13:32.322653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342640+0000","last_change":"2023-10-06T02:46:28.333762+0000","last_active":"2023-10-06T02:46:28.342640+0000","last_peered":"2023-10-06T02:46:28.342640+0000","last_clean":"2023-10-06T02:46:28.342640+0000","last_became_active":"2023-10-06T02:46:28.333193+0000","last_became_peered":"2023-10-06T02:46:28.333193+0000","last_unstale":"2023-10-06T02:46:28.342640+0000","last_undegraded":"2023-10-06T02:46:28.342640+0000","last_fullsized":"2023-10-06T02:46:28.342640+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:53.856632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.570935+0000","last_change":"2023-10-06T02:46:27.557781+0000","last_active":"2023-10-06T02:46:27.570935+0000","last_peered":"2023-10-06T02:46:27.570935+0000","last_clean":"2023-10-06T02:46:27.570935+0000","last_became_active":"2023-10-06T02:46:27.557422+0000","last_became_peered":"2023-10-06T02:46:27.557422+0000","last_unstale":"2023-10-06T02:46:27.570935+0000","last_undegraded":"2023-10-06T02:46:27.570935+0000","last_fullsized":"2023-10-06T02:46:27.570935+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:09.891795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334069+0000","last_change":"2023-10-06T02:46:28.334069+0000","last_active":"2023-10-06T02:46:28.334069+0000","last_peered":"2023-10-06T02:46:28.334069+0000","last_clean":"2023-10-06T02:46:28.334069+0000","last_became_active":"2023-10-06T02:46:28.333761+0000","last_became_peered":"2023-10-06T02:46:28.333761+0000","last_unstale":"2023-10-06T02:46:28.334069+0000","last_undegraded":"2023-10-06T02:46:28.334069+0000","last_fullsized":"2023-10-06T02:46:28.334069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:12.344870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.574306+0000","last_change":"2023-10-06T02:46:27.570952+0000","last_active":"2023-10-06T02:46:28.574306+0000","last_peered":"2023-10-06T02:46:28.574306+0000","last_clean":"2023-10-06T02:46:28.574306+0000","last_became_active":"2023-10-06T02:46:27.567517+0000","last_became_peered":"2023-10-06T02:46:27.567517+0000","last_unstale":"2023-10-06T02:46:28.574306+0000","last_undegraded":"2023-10-06T02:46:28.574306+0000","last_fullsized":"2023-10-06T02:46:28.574306+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:58.977642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.328600+0000","last_change":"2023-10-06T02:46:28.328600+0000","last_active":"2023-10-06T02:46:28.328600+0000","last_peered":"2023-10-06T02:46:28.328600+0000","last_clean":"2023-10-06T02:46:28.328600+0000","last_became_active":"2023-10-06T02:46:28.328160+0000","last_became_peered":"2023-10-06T02:46:28.328160+0000","last_unstale":"2023-10-06T02:46:28.328600+0000","last_undegraded":"2023-10-06T02:46:28.328600+0000","last_fullsized":"2023-10-06T02:46:28.328600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:01.983303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.571051+0000","last_change":"2023-10-06T02:46:27.568939+0000","last_active":"2023-10-06T02:46:27.571051+0000","last_peered":"2023-10-06T02:46:27.571051+0000","last_clean":"2023-10-06T02:46:27.571051+0000","last_became_active":"2023-10-06T02:46:27.568625+0000","last_became_peered":"2023-10-06T02:46:27.568625+0000","last_unstale":"2023-10-06T02:46:27.571051+0000","last_undegraded":"2023-10-06T02:46:27.571051+0000","last_fullsized":"2023-10-06T02:46:27.571051+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:00.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.340639+0000","last_change":"2023-10-06T02:46:28.336965+0000","last_active":"2023-10-06T02:46:28.340639+0000","last_peered":"2023-10-06T02:46:28.340639+0000","last_clean":"2023-10-06T02:46:28.340639+0000","last_became_active":"2023-10-06T02:46:28.336452+0000","last_became_peered":"2023-10-06T02:46:28.336452+0000","last_unstale":"2023-10-06T02:46:28.340639+0000","last_undegraded":"2023-10-06T02:46:28.340639+0000","last_fullsized":"2023-10-06T02:46:28.340639+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:36.444179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574710+0000","last_change":"2023-10-06T02:46:27.568309+0000","last_active":"2023-10-06T02:46:27.574710+0000","last_peered":"2023-10-06T02:46:27.574710+0000","last_clean":"2023-10-06T02:46:27.574710+0000","last_became_active":"2023-10-06T02:46:27.567845+0000","last_became_peered":"2023-10-06T02:46:27.567845+0000","last_unstale":"2023-10-06T02:46:27.574710+0000","last_undegraded":"2023-10-06T02:46:27.574710+0000","last_fullsized":"2023-10-06T02:46:27.574710+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:02.210846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.347154+0000","last_change":"2023-10-06T02:46:28.347154+0000","last_active":"2023-10-06T02:46:28.347154+0000","last_peered":"2023-10-06T02:46:28.347154+0000","last_clean":"2023-10-06T02:46:28.347154+0000","last_became_active":"2023-10-06T02:46:28.344633+0000","last_became_peered":"2023-10-06T02:46:28.344633+0000","last_unstale":"2023-10-06T02:46:28.347154+0000","last_undegraded":"2023-10-06T02:46:28.347154+0000","last_fullsized":"2023-10-06T02:46:28.347154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:59.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572523+0000","last_change":"2023-10-06T02:46:27.568499+0000","last_active":"2023-10-06T02:46:27.572523+0000","last_peered":"2023-10-06T02:46:27.572523+0000","last_clean":"2023-10-06T02:46:27.572523+0000","last_became_active":"2023-10-06T02:46:27.568188+0000","last_became_peered":"2023-10-06T02:46:27.568188+0000","last_unstale":"2023-10-06T02:46:27.572523+0000","last_undegraded":"2023-10-06T02:46:27.572523+0000","last_fullsized":"2023-10-06T02:46:27.572523+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:48.797840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334289+0000","last_change":"2023-10-06T02:46:28.334289+0000","last_active":"2023-10-06T02:46:28.334289+0000","last_peered":"2023-10-06T02:46:28.334289+0000","last_clean":"2023-10-06T02:46:28.334289+0000","last_became_active":"2023-10-06T02:46:28.333373+0000","last_became_peered":"2023-10-06T02:46:28.333373+0000","last_unstale":"2023-10-06T02:46:28.334289+0000","last_undegraded":"2023-10-06T02:46:28.334289+0000","last_fullsized":"2023-10-06T02:46:28.334289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:51.403042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572421+0000","last_change":"2023-10-06T02:46:27.566922+0000","last_active":"2023-10-06T02:46:27.572421+0000","last_peered":"2023-10-06T02:46:27.572421+0000","last_clean":"2023-10-06T02:46:27.572421+0000","last_became_active":"2023-10-06T02:46:27.566623+0000","last_became_peered":"2023-10-06T02:46:27.566623+0000","last_unstale":"2023-10-06T02:46:27.572421+0000","last_undegraded":"2023-10-06T02:46:27.572421+0000","last_fullsized":"2023-10-06T02:46:27.572421+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:09.343776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.322478+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:46:28.322478+0000","last_peered":"2023-10-06T02:46:28.322478+0000","last_clean":"2023-10-06T02:46:28.322478+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:46:28.322478+0000","last_undegraded":"2023-10-06T02:46:28.322478+0000","last_fullsized":"2023-10-06T02:46:28.322478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339073+0000","last_change":"2023-10-06T02:46:28.339073+0000","last_active":"2023-10-06T02:46:28.339073+0000","last_peered":"2023-10-06T02:46:28.339073+0000","last_clean":"2023-10-06T02:46:28.339073+0000","last_became_active":"2023-10-06T02:46:28.338463+0000","last_became_peered":"2023-10-06T02:46:28.338463+0000","last_unstale":"2023-10-06T02:46:28.339073+0000","last_undegraded":"2023-10-06T02:46:28.339073+0000","last_fullsized":"2023-10-06T02:46:28.339073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:13.216359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.320350+0000","last_change":"2023-10-06T02:46:27.571325+0000","last_active":"2023-10-06T02:46:28.320350+0000","last_peered":"2023-10-06T02:46:28.320350+0000","last_clean":"2023-10-06T02:46:28.320350+0000","last_became_active":"2023-10-06T02:46:27.570938+0000","last_became_peered":"2023-10-06T02:46:27.570938+0000","last_unstale":"2023-10-06T02:46:28.320350+0000","last_undegraded":"2023-10-06T02:46:28.320350+0000","last_fullsized":"2023-10-06T02:46:28.320350+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:26.518332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.554220+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:46:27.554220+0000","last_peered":"2023-10-06T02:46:27.554220+0000","last_clean":"2023-10-06T02:46:27.554220+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:46:27.554220+0000","last_undegraded":"2023-10-06T02:46:27.554220+0000","last_fullsized":"2023-10-06T02:46:27.554220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335646+0000","last_change":"2023-10-06T02:46:28.335646+0000","last_active":"2023-10-06T02:46:28.335646+0000","last_peered":"2023-10-06T02:46:28.335646+0000","last_clean":"2023-10-06T02:46:28.335646+0000","last_became_active":"2023-10-06T02:46:28.335354+0000","last_became_peered":"2023-10-06T02:46:28.335354+0000","last_unstale":"2023-10-06T02:46:28.335646+0000","last_undegraded":"2023-10-06T02:46:28.335646+0000","last_fullsized":"2023-10-06T02:46:28.335646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:06.262037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.569268+0000","last_change":"2023-10-06T02:46:27.566156+0000","last_active":"2023-10-06T02:46:27.569268+0000","last_peered":"2023-10-06T02:46:27.569268+0000","last_clean":"2023-10-06T02:46:27.569268+0000","last_became_active":"2023-10-06T02:46:27.565209+0000","last_became_peered":"2023-10-06T02:46:27.565209+0000","last_unstale":"2023-10-06T02:46:27.569268+0000","last_undegraded":"2023-10-06T02:46:27.569268+0000","last_fullsized":"2023-10-06T02:46:27.569268+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:02.210846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.556904+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:46:27.556904+0000","last_peered":"2023-10-06T02:46:27.556904+0000","last_clean":"2023-10-06T02:46:27.556904+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:46:27.556904+0000","last_undegraded":"2023-10-06T02:46:27.556904+0000","last_fullsized":"2023-10-06T02:46:27.556904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339562+0000","last_change":"2023-10-06T02:46:28.336152+0000","last_active":"2023-10-06T02:46:28.339562+0000","last_peered":"2023-10-06T02:46:28.339562+0000","last_clean":"2023-10-06T02:46:28.339562+0000","last_became_active":"2023-10-06T02:46:28.335881+0000","last_became_peered":"2023-10-06T02:46:28.335881+0000","last_unstale":"2023-10-06T02:46:28.339562+0000","last_undegraded":"2023-10-06T02:46:28.339562+0000","last_fullsized":"2023-10-06T02:46:28.339562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:32:32.456718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.581316+0000","last_change":"2023-10-06T02:46:27.575418+0000","last_active":"2023-10-06T02:46:27.581316+0000","last_peered":"2023-10-06T02:46:27.581316+0000","last_clean":"2023-10-06T02:46:27.581316+0000","last_became_active":"2023-10-06T02:46:27.575032+0000","last_became_peered":"2023-10-06T02:46:27.575032+0000","last_unstale":"2023-10-06T02:46:27.581316+0000","last_undegraded":"2023-10-06T02:46:27.581316+0000","last_fullsized":"2023-10-06T02:46:27.581316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:09.343776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.556618+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:46:27.556618+0000","last_peered":"2023-10-06T02:46:27.556618+0000","last_clean":"2023-10-06T02:46:27.556618+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:46:27.556618+0000","last_undegraded":"2023-10-06T02:46:27.556618+0000","last_fullsized":"2023-10-06T02:46:27.556618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342849+0000","last_change":"2023-10-06T02:46:28.337789+0000","last_active":"2023-10-06T02:46:28.342849+0000","last_peered":"2023-10-06T02:46:28.342849+0000","last_clean":"2023-10-06T02:46:28.342849+0000","last_became_active":"2023-10-06T02:46:28.337484+0000","last_became_peered":"2023-10-06T02:46:28.337484+0000","last_unstale":"2023-10-06T02:46:28.342849+0000","last_undegraded":"2023-10-06T02:46:28.342849+0000","last_fullsized":"2023-10-06T02:46:28.342849+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:49.648521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574895+0000","last_change":"2023-10-06T02:46:27.566684+0000","last_active":"2023-10-06T02:46:27.574895+0000","last_peered":"2023-10-06T02:46:27.574895+0000","last_clean":"2023-10-06T02:46:27.574895+0000","last_became_active":"2023-10-06T02:46:27.564717+0000","last_became_peered":"2023-10-06T02:46:27.564717+0000","last_unstale":"2023-10-06T02:46:27.574895+0000","last_undegraded":"2023-10-06T02:46:27.574895+0000","last_fullsized":"2023-10-06T02:46:27.574895+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:25.133829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.564012+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:46:27.564012+0000","last_peered":"2023-10-06T02:46:27.564012+0000","last_clean":"2023-10-06T02:46:27.564012+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:46:27.564012+0000","last_undegraded":"2023-10-06T02:46:27.564012+0000","last_fullsized":"2023-10-06T02:46:27.564012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.333207+0000","last_change":"2023-10-06T02:46:28.333207+0000","last_active":"2023-10-06T02:46:28.333207+0000","last_peered":"2023-10-06T02:46:28.333207+0000","last_clean":"2023-10-06T02:46:28.333207+0000","last_became_active":"2023-10-06T02:46:28.332879+0000","last_became_peered":"2023-10-06T02:46:28.332879+0000","last_unstale":"2023-10-06T02:46:28.333207+0000","last_undegraded":"2023-10-06T02:46:28.333207+0000","last_fullsized":"2023-10-06T02:46:28.333207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:54.979203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572545+0000","last_change":"2023-10-06T02:46:27.568852+0000","last_active":"2023-10-06T02:46:27.572545+0000","last_peered":"2023-10-06T02:46:27.572545+0000","last_clean":"2023-10-06T02:46:27.572545+0000","last_became_active":"2023-10-06T02:46:27.568517+0000","last_became_peered":"2023-10-06T02:46:27.568517+0000","last_unstale":"2023-10-06T02:46:27.572545+0000","last_undegraded":"2023-10-06T02:46:27.572545+0000","last_fullsized":"2023-10-06T02:46:27.572545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:38.294381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.324472+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:46:28.324472+0000","last_peered":"2023-10-06T02:46:28.324472+0000","last_clean":"2023-10-06T02:46:28.324472+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:46:28.324472+0000","last_undegraded":"2023-10-06T02:46:28.324472+0000","last_fullsized":"2023-10-06T02:46:28.324472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.338652+0000","last_change":"2023-10-06T02:46:28.338652+0000","last_active":"2023-10-06T02:46:28.338652+0000","last_peered":"2023-10-06T02:46:28.338652+0000","last_clean":"2023-10-06T02:46:28.338652+0000","last_became_active":"2023-10-06T02:46:28.338321+0000","last_became_peered":"2023-10-06T02:46:28.338321+0000","last_unstale":"2023-10-06T02:46:28.338652+0000","last_undegraded":"2023-10-06T02:46:28.338652+0000","last_fullsized":"2023-10-06T02:46:28.338652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:54.979203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574667+0000","last_change":"2023-10-06T02:46:27.568158+0000","last_active":"2023-10-06T02:46:27.574667+0000","last_peered":"2023-10-06T02:46:27.574667+0000","last_clean":"2023-10-06T02:46:27.574667+0000","last_became_active":"2023-10-06T02:46:27.567886+0000","last_became_peered":"2023-10-06T02:46:27.567886+0000","last_unstale":"2023-10-06T02:46:27.574667+0000","last_undegraded":"2023-10-06T02:46:27.574667+0000","last_fullsized":"2023-10-06T02:46:27.574667+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:05.293970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.552721+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:46:27.552721+0000","last_peered":"2023-10-06T02:46:27.552721+0000","last_clean":"2023-10-06T02:46:27.552721+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:46:27.552721+0000","last_undegraded":"2023-10-06T02:46:27.552721+0000","last_fullsized":"2023-10-06T02:46:27.552721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344918+0000","last_change":"2023-10-06T02:46:28.344918+0000","last_active":"2023-10-06T02:46:28.344918+0000","last_peered":"2023-10-06T02:46:28.344918+0000","last_clean":"2023-10-06T02:46:28.344918+0000","last_became_active":"2023-10-06T02:46:28.344500+0000","last_became_peered":"2023-10-06T02:46:28.344500+0000","last_unstale":"2023-10-06T02:46:28.344918+0000","last_undegraded":"2023-10-06T02:46:28.344918+0000","last_fullsized":"2023-10-06T02:46:28.344918+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:08.877596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572822+0000","last_change":"2023-10-06T02:46:27.574660+0000","last_active":"2023-10-06T02:46:28.572822+0000","last_peered":"2023-10-06T02:46:28.572822+0000","last_clean":"2023-10-06T02:46:28.572822+0000","last_became_active":"2023-10-06T02:46:27.574391+0000","last_became_peered":"2023-10-06T02:46:27.574391+0000","last_unstale":"2023-10-06T02:46:28.572822+0000","last_undegraded":"2023-10-06T02:46:28.572822+0000","last_fullsized":"2023-10-06T02:46:28.572822+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:20.105815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.563846+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:46:27.563846+0000","last_peered":"2023-10-06T02:46:27.563846+0000","last_clean":"2023-10-06T02:46:27.563846+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:46:27.563846+0000","last_undegraded":"2023-10-06T02:46:27.563846+0000","last_fullsized":"2023-10-06T02:46:27.563846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.555148+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:46:27.555148+0000","last_peered":"2023-10-06T02:46:27.555148+0000","last_clean":"2023-10-06T02:46:27.555148+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:46:27.555148+0000","last_undegraded":"2023-10-06T02:46:27.555148+0000","last_fullsized":"2023-10-06T02:46:27.555148+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":810542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.105}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.018},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.92300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.073}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":503808,"data_stored":220728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.98599999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.0960000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.038}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.155},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0740000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.224}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297776,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074064,"statfs":{"total":96636764160,"available":96331841536,"internally_reserved":0,"allocated":499712,"data_stored":220154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.117}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.1830000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.97299999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.304},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.022}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.2909999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.4670000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":221620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.107},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.056}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.996},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.94499999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.282},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.246}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.3779999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.429},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.3320000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":220694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0289999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.874},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.663},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.748}]}]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":221642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81299999999999994}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.012}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.5600000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.331},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1140000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.5029999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297776,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074064,"statfs":{"total":96636764160,"available":96331841536,"internally_reserved":0,"allocated":499712,"data_stored":220154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.751}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.629},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1850000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.274},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0700000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.3140000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1140000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":810486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.05}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.2849999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":1.2070000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:46:31.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:46:31.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 Processor -- start 2023-10-06T02:46:31.684 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- start start 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f502015ace0 0x7f5020161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5020157eb0 con 0x7f50200adb60 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5020069820 con 0x7f5020067970 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5020067ea0 con 0x7f502015ace0 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f502015ace0 0x7f5020161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50200adb60 0x7f50200c6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60772/0 (socket says 172.21.15.33:60772) 2023-10-06T02:46:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501e59c700 1 -- 172.21.15.33:0/246876225 learned_addr learned my addr 172.21.15.33:0/246876225 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501ed9d700 1 -- 172.21.15.33:0/246876225 >> v1:172.21.15.167:6789/0 conn(0x7f5020067970 legacy=0x7f502015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501ed9d700 1 -- 172.21.15.33:0/246876225 >> v1:172.21.15.33:6789/0 conn(0x7f50200adb60 legacy=0x7f50200c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501dd9b700 1 -- 172.21.15.33:0/246876225 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762396512 0 0) 0x7f5010001110 con 0x7f50200adb60 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501ed9d700 1 -- 172.21.15.33:0/246876225 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020068180 con 0x7f502015ace0 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501ed9d700 1 --2- 172.21.15.33:0/246876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f5020161970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5018009e60 tx=0x7f5018009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fb82a998913d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501dd9b700 1 -- 172.21.15.33:0/246876225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800e070 con 0x7f502015ace0 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501dd9b700 1 -- 172.21.15.33:0/246876225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501800bc30 con 0x7f502015ace0 2023-10-06T02:46:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501dd9b700 1 -- 172.21.15.33:0/246876225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018012430 con 0x7f502015ace0 2023-10-06T02:46:31.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- 172.21.15.33:0/246876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 msgr2=0x7f5020161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 --2- 172.21.15.33:0/246876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f5020161970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5018009e60 tx=0x7f5018009780 comp rx=0 tx=0).stop 2023-10-06T02:46:31.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- 172.21.15.33:0/246876225 shutdown_connections 2023-10-06T02:46:31.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 --2- 172.21.15.33:0/246876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f5020161970 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- 172.21.15.33:0/246876225 >> 172.21.15.33:0/246876225 conn(0x7f50200c8240 msgr2=0x7f50200c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- 172.21.15.33:0/246876225 shutdown_connections 2023-10-06T02:46:31.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.681+0000 7f501f59e700 1 -- 172.21.15.33:0/246876225 wait complete. 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 Processor -- start 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 -- start start 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5020067970 0x7f5020169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50200adb60 0x7f50201556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f50200a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50201694f0 con 0x7f502015ace0 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020168f10 con 0x7f5020067970 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5020169ec0 con 0x7f50200adb60 2023-10-06T02:46:31.691 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f50200a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5020067970 0x7f5020169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f50200a88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36078/0 (socket says 172.21.15.33:36078) 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 -- 172.21.15.33:0/3721526077 learned_addr learned my addr 172.21.15.33:0/3721526077 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f5015d9b700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50200adb60 0x7f50201556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 -- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50200adb60 msgr2=0x7f50201556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50200adb60 0x7f50201556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 -- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5020067970 msgr2=0x7f5020169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5020067970 0x7f5020169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 -- 172.21.15.33:0/3721526077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018009a20 con 0x7f502015ace0 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501e59c700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5020067970 0x7f5020169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:31.692 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501ed9d700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f50200a88d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5018013000 tx=0x7f5018019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018027470 con 0x7f502015ace0 2023-10-06T02:46:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50180275d0 con 0x7f502015ace0 2023-10-06T02:46:31.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.685+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50200c5d00 con 0x7f502015ace0 2023-10-06T02:46:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018027990 con 0x7f502015ace0 2023-10-06T02:46:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5020169180 con 0x7f502015ace0 2023-10-06T02:46:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5020169660 con 0x7f502015ace0 2023-10-06T02:46:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5018024070 con 0x7f502015ace0 2023-10-06T02:46:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f5016ffd700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5000060580 0x7f5000062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f501807f370 con 0x7f502015ace0 2023-10-06T02:46:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.689+0000 7f501e59c700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5000060580 0x7f5000062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.693+0000 7f501e59c700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5000060580 0x7f5000062a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5010006920 tx=0x7f5010007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.701+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f501804f620 con 0x7f502015ace0 2023-10-06T02:46:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.861+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5020071040 con 0x7f5000060580 2023-10-06T02:46:31.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.877+0000 7f5016ffd700 1 -- 172.21.15.33:0/3721526077 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309598 (secure 0 0 0) 0x7f5020071040 con 0x7f5000060580 2023-10-06T02:46:31.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5000060580 msgr2=0x7f5000062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5000060580 0x7f5000062a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5010006920 tx=0x7f5010007b90 comp rx=0 tx=0).stop 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 msgr2=0x7f50200a88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f50200a88d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5018013000 tx=0x7f5018019000 comp rx=0 tx=0).stop 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 shutdown_connections 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5000060580 0x7f5000062a30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50200adb60 0x7f50201556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502015ace0 0x7f50200a88d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 --2- 172.21.15.33:0/3721526077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5020067970 0x7f5020169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 >> 172.21.15.33:0/3721526077 conn(0x7f50200c8240 msgr2=0x7f50200b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 shutdown_connections 2023-10-06T02:46:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:31.881+0000 7f501f59e700 1 -- 172.21.15.33:0/3721526077 wait complete. 2023-10-06T02:46:31.890 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:46:31.900 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-10-06T02:46:30.582947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383400,"kb_used_data":5096,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591320,"statfs":{"total":773094113280,"available":770653511680,"internally_reserved":0,"allocated":5218304,"data_stored":2946020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.051282"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335955+0000","last_change":"2023-10-06T02:46:28.335955+0000","last_active":"2023-10-06T02:46:28.335955+0000","last_peered":"2023-10-06T02:46:28.335955+0000","last_clean":"2023-10-06T02:46:28.335955+0000","last_became_active":"2023-10-06T02:46:28.335340+0000","last_became_peered":"2023-10-06T02:46:28.335340+0000","last_unstale":"2023-10-06T02:46:28.335955+0000","last_undegraded":"2023-10-06T02:46:28.335955+0000","last_fullsized":"2023-10-06T02:46:28.335955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:36.931513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339465+0000","last_change":"2023-10-06T02:46:28.339465+0000","last_active":"2023-10-06T02:46:28.339465+0000","last_peered":"2023-10-06T02:46:28.339465+0000","last_clean":"2023-10-06T02:46:28.339465+0000","last_became_active":"2023-10-06T02:46:28.339157+0000","last_became_peered":"2023-10-06T02:46:28.339157+0000","last_unstale":"2023-10-06T02:46:28.339465+0000","last_undegraded":"2023-10-06T02:46:28.339465+0000","last_fullsized":"2023-10-06T02:46:28.339465+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:27.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339508+0000","last_change":"2023-10-06T02:46:28.334929+0000","last_active":"2023-10-06T02:46:28.339508+0000","last_peered":"2023-10-06T02:46:28.339508+0000","last_clean":"2023-10-06T02:46:28.339508+0000","last_became_active":"2023-10-06T02:46:28.334653+0000","last_became_peered":"2023-10-06T02:46:28.334653+0000","last_unstale":"2023-10-06T02:46:28.339508+0000","last_undegraded":"2023-10-06T02:46:28.339508+0000","last_fullsized":"2023-10-06T02:46:28.339508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:17:30.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.347217+0000","last_change":"2023-10-06T02:46:28.347217+0000","last_active":"2023-10-06T02:46:28.347217+0000","last_peered":"2023-10-06T02:46:28.347217+0000","last_clean":"2023-10-06T02:46:28.347217+0000","last_became_active":"2023-10-06T02:46:28.344294+0000","last_became_peered":"2023-10-06T02:46:28.344294+0000","last_unstale":"2023-10-06T02:46:28.347217+0000","last_undegraded":"2023-10-06T02:46:28.347217+0000","last_fullsized":"2023-10-06T02:46:28.347217+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:08.063199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.337687+0000","last_change":"2023-10-06T02:46:28.337687+0000","last_active":"2023-10-06T02:46:28.337687+0000","last_peered":"2023-10-06T02:46:28.337687+0000","last_clean":"2023-10-06T02:46:28.337687+0000","last_became_active":"2023-10-06T02:46:28.337391+0000","last_became_peered":"2023-10-06T02:46:28.337391+0000","last_unstale":"2023-10-06T02:46:28.337687+0000","last_undegraded":"2023-10-06T02:46:28.337687+0000","last_fullsized":"2023-10-06T02:46:28.337687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.111686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.337810+0000","last_change":"2023-10-06T02:46:28.337810+0000","last_active":"2023-10-06T02:46:28.337810+0000","last_peered":"2023-10-06T02:46:28.337810+0000","last_clean":"2023-10-06T02:46:28.337810+0000","last_became_active":"2023-10-06T02:46:28.335855+0000","last_became_peered":"2023-10-06T02:46:28.335855+0000","last_unstale":"2023-10-06T02:46:28.337810+0000","last_undegraded":"2023-10-06T02:46:28.337810+0000","last_fullsized":"2023-10-06T02:46:28.337810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:34.618192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335300+0000","last_change":"2023-10-06T02:46:28.335300+0000","last_active":"2023-10-06T02:46:28.335300+0000","last_peered":"2023-10-06T02:46:28.335300+0000","last_clean":"2023-10-06T02:46:28.335300+0000","last_became_active":"2023-10-06T02:46:28.334861+0000","last_became_peered":"2023-10-06T02:46:28.334861+0000","last_unstale":"2023-10-06T02:46:28.335300+0000","last_undegraded":"2023-10-06T02:46:28.335300+0000","last_fullsized":"2023-10-06T02:46:28.335300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:21.922252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344494+0000","last_change":"2023-10-06T02:46:28.344494+0000","last_active":"2023-10-06T02:46:28.344494+0000","last_peered":"2023-10-06T02:46:28.344494+0000","last_clean":"2023-10-06T02:46:28.344494+0000","last_became_active":"2023-10-06T02:46:28.341901+0000","last_became_peered":"2023-10-06T02:46:28.341901+0000","last_unstale":"2023-10-06T02:46:28.344494+0000","last_undegraded":"2023-10-06T02:46:28.344494+0000","last_fullsized":"2023-10-06T02:46:28.344494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:16.888525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.348058+0000","last_change":"2023-10-06T02:46:28.348058+0000","last_active":"2023-10-06T02:46:28.348058+0000","last_peered":"2023-10-06T02:46:28.348058+0000","last_clean":"2023-10-06T02:46:28.348058+0000","last_became_active":"2023-10-06T02:46:28.347744+0000","last_became_peered":"2023-10-06T02:46:28.347744+0000","last_unstale":"2023-10-06T02:46:28.348058+0000","last_undegraded":"2023-10-06T02:46:28.348058+0000","last_fullsized":"2023-10-06T02:46:28.348058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:06.262037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341686+0000","last_change":"2023-10-06T02:46:28.341686+0000","last_active":"2023-10-06T02:46:28.341686+0000","last_peered":"2023-10-06T02:46:28.341686+0000","last_clean":"2023-10-06T02:46:28.341686+0000","last_became_active":"2023-10-06T02:46:28.340990+0000","last_became_peered":"2023-10-06T02:46:28.340990+0000","last_unstale":"2023-10-06T02:46:28.341686+0000","last_undegraded":"2023-10-06T02:46:28.341686+0000","last_fullsized":"2023-10-06T02:46:28.341686+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:16.514623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344398+0000","last_change":"2023-10-06T02:46:28.344398+0000","last_active":"2023-10-06T02:46:28.344398+0000","last_peered":"2023-10-06T02:46:28.344398+0000","last_clean":"2023-10-06T02:46:28.344398+0000","last_became_active":"2023-10-06T02:46:28.344068+0000","last_became_peered":"2023-10-06T02:46:28.344068+0000","last_unstale":"2023-10-06T02:46:28.344398+0000","last_undegraded":"2023-10-06T02:46:28.344398+0000","last_fullsized":"2023-10-06T02:46:28.344398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:04.203288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.337245+0000","last_change":"2023-10-06T02:46:28.337245+0000","last_active":"2023-10-06T02:46:28.337245+0000","last_peered":"2023-10-06T02:46:28.337245+0000","last_clean":"2023-10-06T02:46:28.337245+0000","last_became_active":"2023-10-06T02:46:28.336773+0000","last_became_peered":"2023-10-06T02:46:28.336773+0000","last_unstale":"2023-10-06T02:46:28.337245+0000","last_undegraded":"2023-10-06T02:46:28.337245+0000","last_fullsized":"2023-10-06T02:46:28.337245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:16.514623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.345791+0000","last_change":"2023-10-06T02:46:28.338959+0000","last_active":"2023-10-06T02:46:28.345791+0000","last_peered":"2023-10-06T02:46:28.345791+0000","last_clean":"2023-10-06T02:46:28.345791+0000","last_became_active":"2023-10-06T02:46:28.338610+0000","last_became_peered":"2023-10-06T02:46:28.338610+0000","last_unstale":"2023-10-06T02:46:28.345791+0000","last_undegraded":"2023-10-06T02:46:28.345791+0000","last_fullsized":"2023-10-06T02:46:28.345791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:18.705652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344964+0000","last_change":"2023-10-06T02:46:28.344964+0000","last_active":"2023-10-06T02:46:28.344964+0000","last_peered":"2023-10-06T02:46:28.344964+0000","last_clean":"2023-10-06T02:46:28.344964+0000","last_became_active":"2023-10-06T02:46:28.344116+0000","last_became_peered":"2023-10-06T02:46:28.344116+0000","last_unstale":"2023-10-06T02:46:28.344964+0000","last_undegraded":"2023-10-06T02:46:28.344964+0000","last_fullsized":"2023-10-06T02:46:28.344964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:26.287253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341259+0000","last_change":"2023-10-06T02:46:28.341259+0000","last_active":"2023-10-06T02:46:28.341259+0000","last_peered":"2023-10-06T02:46:28.341259+0000","last_clean":"2023-10-06T02:46:28.341259+0000","last_became_active":"2023-10-06T02:46:28.340989+0000","last_became_peered":"2023-10-06T02:46:28.340989+0000","last_unstale":"2023-10-06T02:46:28.341259+0000","last_undegraded":"2023-10-06T02:46:28.341259+0000","last_fullsized":"2023-10-06T02:46:28.341259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:27.498868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.346702+0000","last_change":"2023-10-06T02:46:28.344150+0000","last_active":"2023-10-06T02:46:28.346702+0000","last_peered":"2023-10-06T02:46:28.346702+0000","last_clean":"2023-10-06T02:46:28.346702+0000","last_became_active":"2023-10-06T02:46:28.343816+0000","last_became_peered":"2023-10-06T02:46:28.343816+0000","last_unstale":"2023-10-06T02:46:28.346702+0000","last_undegraded":"2023-10-06T02:46:28.346702+0000","last_fullsized":"2023-10-06T02:46:28.346702+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:58.728344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.330309+0000","last_change":"2023-10-06T02:46:28.328175+0000","last_active":"2023-10-06T02:46:28.330309+0000","last_peered":"2023-10-06T02:46:28.330309+0000","last_clean":"2023-10-06T02:46:28.330309+0000","last_became_active":"2023-10-06T02:46:28.327825+0000","last_became_peered":"2023-10-06T02:46:28.327825+0000","last_unstale":"2023-10-06T02:46:28.330309+0000","last_undegraded":"2023-10-06T02:46:28.330309+0000","last_fullsized":"2023-10-06T02:46:28.330309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:47.028557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342417+0000","last_change":"2023-10-06T02:46:28.338351+0000","last_active":"2023-10-06T02:46:28.342417+0000","last_peered":"2023-10-06T02:46:28.342417+0000","last_clean":"2023-10-06T02:46:28.342417+0000","last_became_active":"2023-10-06T02:46:28.337909+0000","last_became_peered":"2023-10-06T02:46:28.337909+0000","last_unstale":"2023-10-06T02:46:28.342417+0000","last_undegraded":"2023-10-06T02:46:28.342417+0000","last_fullsized":"2023-10-06T02:46:28.342417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:46.909260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342832+0000","last_change":"2023-10-06T02:46:28.337692+0000","last_active":"2023-10-06T02:46:28.342832+0000","last_peered":"2023-10-06T02:46:28.342832+0000","last_clean":"2023-10-06T02:46:28.342832+0000","last_became_active":"2023-10-06T02:46:28.337393+0000","last_became_peered":"2023-10-06T02:46:28.337393+0000","last_unstale":"2023-10-06T02:46:28.342832+0000","last_undegraded":"2023-10-06T02:46:28.342832+0000","last_fullsized":"2023-10-06T02:46:28.342832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:10.904244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342088+0000","last_change":"2023-10-06T02:46:28.337814+0000","last_active":"2023-10-06T02:46:28.342088+0000","last_peered":"2023-10-06T02:46:28.342088+0000","last_clean":"2023-10-06T02:46:28.342088+0000","last_became_active":"2023-10-06T02:46:28.337511+0000","last_became_peered":"2023-10-06T02:46:28.337511+0000","last_unstale":"2023-10-06T02:46:28.342088+0000","last_undegraded":"2023-10-06T02:46:28.342088+0000","last_fullsized":"2023-10-06T02:46:28.342088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:07.273611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342495+0000","last_change":"2023-10-06T02:46:28.342495+0000","last_active":"2023-10-06T02:46:28.342495+0000","last_peered":"2023-10-06T02:46:28.342495+0000","last_clean":"2023-10-06T02:46:28.342495+0000","last_became_active":"2023-10-06T02:46:28.342106+0000","last_became_peered":"2023-10-06T02:46:28.342106+0000","last_unstale":"2023-10-06T02:46:28.342495+0000","last_undegraded":"2023-10-06T02:46:28.342495+0000","last_fullsized":"2023-10-06T02:46:28.342495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:07.886293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341380+0000","last_change":"2023-10-06T02:46:28.341380+0000","last_active":"2023-10-06T02:46:28.341380+0000","last_peered":"2023-10-06T02:46:28.341380+0000","last_clean":"2023-10-06T02:46:28.341380+0000","last_became_active":"2023-10-06T02:46:28.340902+0000","last_became_peered":"2023-10-06T02:46:28.340902+0000","last_unstale":"2023-10-06T02:46:28.341380+0000","last_undegraded":"2023-10-06T02:46:28.341380+0000","last_fullsized":"2023-10-06T02:46:28.341380+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:43.589676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342726+0000","last_change":"2023-10-06T02:46:28.336390+0000","last_active":"2023-10-06T02:46:28.342726+0000","last_peered":"2023-10-06T02:46:28.342726+0000","last_clean":"2023-10-06T02:46:28.342726+0000","last_became_active":"2023-10-06T02:46:28.335889+0000","last_became_peered":"2023-10-06T02:46:28.335889+0000","last_unstale":"2023-10-06T02:46:28.342726+0000","last_undegraded":"2023-10-06T02:46:28.342726+0000","last_fullsized":"2023-10-06T02:46:28.342726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:07.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342769+0000","last_change":"2023-10-06T02:46:28.337178+0000","last_active":"2023-10-06T02:46:28.342769+0000","last_peered":"2023-10-06T02:46:28.342769+0000","last_clean":"2023-10-06T02:46:28.342769+0000","last_became_active":"2023-10-06T02:46:28.336876+0000","last_became_peered":"2023-10-06T02:46:28.336876+0000","last_unstale":"2023-10-06T02:46:28.342769+0000","last_undegraded":"2023-10-06T02:46:28.342769+0000","last_fullsized":"2023-10-06T02:46:28.342769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.678648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342563+0000","last_change":"2023-10-06T02:46:28.332897+0000","last_active":"2023-10-06T02:46:28.342563+0000","last_peered":"2023-10-06T02:46:28.342563+0000","last_clean":"2023-10-06T02:46:28.342563+0000","last_became_active":"2023-10-06T02:46:28.332573+0000","last_became_peered":"2023-10-06T02:46:28.332573+0000","last_unstale":"2023-10-06T02:46:28.342563+0000","last_undegraded":"2023-10-06T02:46:28.342563+0000","last_fullsized":"2023-10-06T02:46:28.342563+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:10.254146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.346118+0000","last_change":"2023-10-06T02:46:28.339816+0000","last_active":"2023-10-06T02:46:28.346118+0000","last_peered":"2023-10-06T02:46:28.346118+0000","last_clean":"2023-10-06T02:46:28.346118+0000","last_became_active":"2023-10-06T02:46:28.338734+0000","last_became_peered":"2023-10-06T02:46:28.338734+0000","last_unstale":"2023-10-06T02:46:28.346118+0000","last_undegraded":"2023-10-06T02:46:28.346118+0000","last_fullsized":"2023-10-06T02:46:28.346118+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:03.223296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.330460+0000","last_change":"2023-10-06T02:46:28.329055+0000","last_active":"2023-10-06T02:46:28.330460+0000","last_peered":"2023-10-06T02:46:28.330460+0000","last_clean":"2023-10-06T02:46:28.330460+0000","last_became_active":"2023-10-06T02:46:28.328774+0000","last_became_peered":"2023-10-06T02:46:28.328774+0000","last_unstale":"2023-10-06T02:46:28.330460+0000","last_undegraded":"2023-10-06T02:46:28.330460+0000","last_fullsized":"2023-10-06T02:46:28.330460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:10.157128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334636+0000","last_change":"2023-10-06T02:46:28.334636+0000","last_active":"2023-10-06T02:46:28.334636+0000","last_peered":"2023-10-06T02:46:28.334636+0000","last_clean":"2023-10-06T02:46:28.334636+0000","last_became_active":"2023-10-06T02:46:28.334265+0000","last_became_peered":"2023-10-06T02:46:28.334265+0000","last_unstale":"2023-10-06T02:46:28.334636+0000","last_undegraded":"2023-10-06T02:46:28.334636+0000","last_fullsized":"2023-10-06T02:46:28.334636+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:43.589676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342579+0000","last_change":"2023-10-06T02:46:28.333000+0000","last_active":"2023-10-06T02:46:28.342579+0000","last_peered":"2023-10-06T02:46:28.342579+0000","last_clean":"2023-10-06T02:46:28.342579+0000","last_became_active":"2023-10-06T02:46:28.332673+0000","last_became_peered":"2023-10-06T02:46:28.332673+0000","last_unstale":"2023-10-06T02:46:28.342579+0000","last_undegraded":"2023-10-06T02:46:28.342579+0000","last_fullsized":"2023-10-06T02:46:28.342579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:30.592850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339600+0000","last_change":"2023-10-06T02:46:28.335854+0000","last_active":"2023-10-06T02:46:28.339600+0000","last_peered":"2023-10-06T02:46:28.339600+0000","last_clean":"2023-10-06T02:46:28.339600+0000","last_became_active":"2023-10-06T02:46:28.335565+0000","last_became_peered":"2023-10-06T02:46:28.335565+0000","last_unstale":"2023-10-06T02:46:28.339600+0000","last_undegraded":"2023-10-06T02:46:28.339600+0000","last_fullsized":"2023-10-06T02:46:28.339600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:07.794047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.338585+0000","last_change":"2023-10-06T02:46:28.338585+0000","last_active":"2023-10-06T02:46:28.338585+0000","last_peered":"2023-10-06T02:46:28.338585+0000","last_clean":"2023-10-06T02:46:28.338585+0000","last_became_active":"2023-10-06T02:46:28.335815+0000","last_became_peered":"2023-10-06T02:46:28.335815+0000","last_unstale":"2023-10-06T02:46:28.338585+0000","last_undegraded":"2023-10-06T02:46:28.338585+0000","last_fullsized":"2023-10-06T02:46:28.338585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:44.775700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.346319+0000","last_change":"2023-10-06T02:46:28.343557+0000","last_active":"2023-10-06T02:46:28.346319+0000","last_peered":"2023-10-06T02:46:28.346319+0000","last_clean":"2023-10-06T02:46:28.346319+0000","last_became_active":"2023-10-06T02:46:28.343239+0000","last_became_peered":"2023-10-06T02:46:28.343239+0000","last_unstale":"2023-10-06T02:46:28.346319+0000","last_undegraded":"2023-10-06T02:46:28.346319+0000","last_fullsized":"2023-10-06T02:46:28.346319+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:06.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335227+0000","last_change":"2023-10-06T02:46:28.335227+0000","last_active":"2023-10-06T02:46:28.335227+0000","last_peered":"2023-10-06T02:46:28.335227+0000","last_clean":"2023-10-06T02:46:28.335227+0000","last_became_active":"2023-10-06T02:46:28.334935+0000","last_became_peered":"2023-10-06T02:46:28.334935+0000","last_unstale":"2023-10-06T02:46:28.335227+0000","last_undegraded":"2023-10-06T02:46:28.335227+0000","last_fullsized":"2023-10-06T02:46:28.335227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:05.138628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.573580+0000","last_change":"2023-10-06T02:46:27.557700+0000","last_active":"2023-10-06T02:46:28.573580+0000","last_peered":"2023-10-06T02:46:28.573580+0000","last_clean":"2023-10-06T02:46:28.573580+0000","last_became_active":"2023-10-06T02:46:27.557396+0000","last_became_peered":"2023-10-06T02:46:27.557396+0000","last_unstale":"2023-10-06T02:46:28.573580+0000","last_undegraded":"2023-10-06T02:46:28.573580+0000","last_fullsized":"2023-10-06T02:46:28.573580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:15.146702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.340191+0000","last_change":"2023-10-06T02:46:28.329957+0000","last_active":"2023-10-06T02:46:28.340191+0000","last_peered":"2023-10-06T02:46:28.340191+0000","last_clean":"2023-10-06T02:46:28.340191+0000","last_became_active":"2023-10-06T02:46:28.329675+0000","last_became_peered":"2023-10-06T02:46:28.329675+0000","last_unstale":"2023-10-06T02:46:28.340191+0000","last_undegraded":"2023-10-06T02:46:28.340191+0000","last_fullsized":"2023-10-06T02:46:28.340191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:04.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572595+0000","last_change":"2023-10-06T02:46:27.561770+0000","last_active":"2023-10-06T02:46:28.572595+0000","last_peered":"2023-10-06T02:46:28.572595+0000","last_clean":"2023-10-06T02:46:28.572595+0000","last_became_active":"2023-10-06T02:46:27.561473+0000","last_became_peered":"2023-10-06T02:46:27.561473+0000","last_unstale":"2023-10-06T02:46:28.572595+0000","last_undegraded":"2023-10-06T02:46:28.572595+0000","last_fullsized":"2023-10-06T02:46:28.572595+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:35.919063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.338447+0000","last_change":"2023-10-06T02:46:28.338447+0000","last_active":"2023-10-06T02:46:28.338447+0000","last_peered":"2023-10-06T02:46:28.338447+0000","last_clean":"2023-10-06T02:46:28.338447+0000","last_became_active":"2023-10-06T02:46:28.338137+0000","last_became_peered":"2023-10-06T02:46:28.338137+0000","last_unstale":"2023-10-06T02:46:28.338447+0000","last_undegraded":"2023-10-06T02:46:28.338447+0000","last_fullsized":"2023-10-06T02:46:28.338447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:54.979203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.570579+0000","last_change":"2023-10-06T02:46:27.568255+0000","last_active":"2023-10-06T02:46:28.570579+0000","last_peered":"2023-10-06T02:46:28.570579+0000","last_clean":"2023-10-06T02:46:28.570579+0000","last_became_active":"2023-10-06T02:46:27.567918+0000","last_became_peered":"2023-10-06T02:46:27.567918+0000","last_unstale":"2023-10-06T02:46:28.570579+0000","last_undegraded":"2023-10-06T02:46:28.570579+0000","last_fullsized":"2023-10-06T02:46:28.570579+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:05.364568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.331925+0000","last_change":"2023-10-06T02:46:28.331925+0000","last_active":"2023-10-06T02:46:28.331925+0000","last_peered":"2023-10-06T02:46:28.331925+0000","last_clean":"2023-10-06T02:46:28.331925+0000","last_became_active":"2023-10-06T02:46:28.331622+0000","last_became_peered":"2023-10-06T02:46:28.331622+0000","last_unstale":"2023-10-06T02:46:28.331925+0000","last_undegraded":"2023-10-06T02:46:28.331925+0000","last_fullsized":"2023-10-06T02:46:28.331925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:39.306831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.571503+0000","last_change":"2023-10-06T02:46:27.565565+0000","last_active":"2023-10-06T02:46:28.571503+0000","last_peered":"2023-10-06T02:46:28.571503+0000","last_clean":"2023-10-06T02:46:28.571503+0000","last_became_active":"2023-10-06T02:46:27.565189+0000","last_became_peered":"2023-10-06T02:46:27.565189+0000","last_unstale":"2023-10-06T02:46:28.571503+0000","last_undegraded":"2023-10-06T02:46:28.571503+0000","last_fullsized":"2023-10-06T02:46:28.571503+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:49.647098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.330369+0000","last_change":"2023-10-06T02:46:28.328357+0000","last_active":"2023-10-06T02:46:28.330369+0000","last_peered":"2023-10-06T02:46:28.330369+0000","last_clean":"2023-10-06T02:46:28.330369+0000","last_became_active":"2023-10-06T02:46:28.328001+0000","last_became_peered":"2023-10-06T02:46:28.328001+0000","last_unstale":"2023-10-06T02:46:28.330369+0000","last_undegraded":"2023-10-06T02:46:28.330369+0000","last_fullsized":"2023-10-06T02:46:28.330369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:59.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.327552+0000","last_change":"2023-10-06T02:46:27.570497+0000","last_active":"2023-10-06T02:46:28.327552+0000","last_peered":"2023-10-06T02:46:28.327552+0000","last_clean":"2023-10-06T02:46:28.327552+0000","last_became_active":"2023-10-06T02:46:27.567022+0000","last_became_peered":"2023-10-06T02:46:27.567022+0000","last_unstale":"2023-10-06T02:46:28.327552+0000","last_undegraded":"2023-10-06T02:46:28.327552+0000","last_fullsized":"2023-10-06T02:46:28.327552+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:29.323681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335169+0000","last_change":"2023-10-06T02:46:28.335169+0000","last_active":"2023-10-06T02:46:28.335169+0000","last_peered":"2023-10-06T02:46:28.335169+0000","last_clean":"2023-10-06T02:46:28.335169+0000","last_became_active":"2023-10-06T02:46:28.334613+0000","last_became_peered":"2023-10-06T02:46:28.334613+0000","last_unstale":"2023-10-06T02:46:28.335169+0000","last_undegraded":"2023-10-06T02:46:28.335169+0000","last_fullsized":"2023-10-06T02:46:28.335169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:46.909260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.570456+0000","last_change":"2023-10-06T02:46:27.567470+0000","last_active":"2023-10-06T02:46:28.570456+0000","last_peered":"2023-10-06T02:46:28.570456+0000","last_clean":"2023-10-06T02:46:28.570456+0000","last_became_active":"2023-10-06T02:46:27.567183+0000","last_became_peered":"2023-10-06T02:46:27.567183+0000","last_unstale":"2023-10-06T02:46:28.570456+0000","last_undegraded":"2023-10-06T02:46:28.570456+0000","last_fullsized":"2023-10-06T02:46:28.570456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:20.909802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.341494+0000","last_change":"2023-10-06T02:46:28.341494+0000","last_active":"2023-10-06T02:46:28.341494+0000","last_peered":"2023-10-06T02:46:28.341494+0000","last_clean":"2023-10-06T02:46:28.341494+0000","last_became_active":"2023-10-06T02:46:28.340853+0000","last_became_peered":"2023-10-06T02:46:28.340853+0000","last_unstale":"2023-10-06T02:46:28.341494+0000","last_undegraded":"2023-10-06T02:46:28.341494+0000","last_fullsized":"2023-10-06T02:46:28.341494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.111686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.570267+0000","last_change":"2023-10-06T02:46:27.567473+0000","last_active":"2023-10-06T02:46:27.570267+0000","last_peered":"2023-10-06T02:46:27.570267+0000","last_clean":"2023-10-06T02:46:27.570267+0000","last_became_active":"2023-10-06T02:46:27.567163+0000","last_became_peered":"2023-10-06T02:46:27.567163+0000","last_unstale":"2023-10-06T02:46:27.570267+0000","last_undegraded":"2023-10-06T02:46:27.570267+0000","last_fullsized":"2023-10-06T02:46:27.570267+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:56.478162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342707+0000","last_change":"2023-10-06T02:46:28.336189+0000","last_active":"2023-10-06T02:46:28.342707+0000","last_peered":"2023-10-06T02:46:28.342707+0000","last_clean":"2023-10-06T02:46:28.342707+0000","last_became_active":"2023-10-06T02:46:28.335487+0000","last_became_peered":"2023-10-06T02:46:28.335487+0000","last_unstale":"2023-10-06T02:46:28.342707+0000","last_undegraded":"2023-10-06T02:46:28.342707+0000","last_fullsized":"2023-10-06T02:46:28.342707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:45.410932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.566975+0000","last_change":"2023-10-06T02:46:27.564075+0000","last_active":"2023-10-06T02:46:27.566975+0000","last_peered":"2023-10-06T02:46:27.566975+0000","last_clean":"2023-10-06T02:46:27.566975+0000","last_became_active":"2023-10-06T02:46:27.563766+0000","last_became_peered":"2023-10-06T02:46:27.563766+0000","last_unstale":"2023-10-06T02:46:27.566975+0000","last_undegraded":"2023-10-06T02:46:27.566975+0000","last_fullsized":"2023-10-06T02:46:27.566975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:35.287886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342397+0000","last_change":"2023-10-06T02:46:28.338161+0000","last_active":"2023-10-06T02:46:28.342397+0000","last_peered":"2023-10-06T02:46:28.342397+0000","last_clean":"2023-10-06T02:46:28.342397+0000","last_became_active":"2023-10-06T02:46:28.337552+0000","last_became_peered":"2023-10-06T02:46:28.337552+0000","last_unstale":"2023-10-06T02:46:28.342397+0000","last_undegraded":"2023-10-06T02:46:28.342397+0000","last_fullsized":"2023-10-06T02:46:28.342397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:04.203288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.320395+0000","last_change":"2023-10-06T02:46:27.566209+0000","last_active":"2023-10-06T02:46:28.320395+0000","last_peered":"2023-10-06T02:46:28.320395+0000","last_clean":"2023-10-06T02:46:28.320395+0000","last_became_active":"2023-10-06T02:46:27.565930+0000","last_became_peered":"2023-10-06T02:46:27.565930+0000","last_unstale":"2023-10-06T02:46:28.320395+0000","last_undegraded":"2023-10-06T02:46:28.320395+0000","last_fullsized":"2023-10-06T02:46:28.320395+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:25.274803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.336186+0000","last_change":"2023-10-06T02:46:28.336186+0000","last_active":"2023-10-06T02:46:28.336186+0000","last_peered":"2023-10-06T02:46:28.336186+0000","last_clean":"2023-10-06T02:46:28.336186+0000","last_became_active":"2023-10-06T02:46:28.335669+0000","last_became_peered":"2023-10-06T02:46:28.335669+0000","last_unstale":"2023-10-06T02:46:28.336186+0000","last_undegraded":"2023-10-06T02:46:28.336186+0000","last_fullsized":"2023-10-06T02:46:28.336186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:06.262037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574550+0000","last_change":"2023-10-06T02:46:27.568986+0000","last_active":"2023-10-06T02:46:27.574550+0000","last_peered":"2023-10-06T02:46:27.574550+0000","last_clean":"2023-10-06T02:46:27.574550+0000","last_became_active":"2023-10-06T02:46:27.568702+0000","last_became_peered":"2023-10-06T02:46:27.568702+0000","last_unstale":"2023-10-06T02:46:27.574550+0000","last_undegraded":"2023-10-06T02:46:27.574550+0000","last_fullsized":"2023-10-06T02:46:27.574550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:35.431730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339031+0000","last_change":"2023-10-06T02:46:28.339031+0000","last_active":"2023-10-06T02:46:28.339031+0000","last_peered":"2023-10-06T02:46:28.339031+0000","last_clean":"2023-10-06T02:46:28.339031+0000","last_became_active":"2023-10-06T02:46:28.338375+0000","last_became_peered":"2023-10-06T02:46:28.338375+0000","last_unstale":"2023-10-06T02:46:28.339031+0000","last_undegraded":"2023-10-06T02:46:28.339031+0000","last_fullsized":"2023-10-06T02:46:28.339031+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:25.554675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572741+0000","last_change":"2023-10-06T02:46:27.568485+0000","last_active":"2023-10-06T02:46:28.572741+0000","last_peered":"2023-10-06T02:46:28.572741+0000","last_clean":"2023-10-06T02:46:28.572741+0000","last_became_active":"2023-10-06T02:46:27.567854+0000","last_became_peered":"2023-10-06T02:46:27.567854+0000","last_unstale":"2023-10-06T02:46:28.572741+0000","last_undegraded":"2023-10-06T02:46:28.572741+0000","last_fullsized":"2023-10-06T02:46:28.572741+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:00.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334936+0000","last_change":"2023-10-06T02:46:28.334936+0000","last_active":"2023-10-06T02:46:28.334936+0000","last_peered":"2023-10-06T02:46:28.334936+0000","last_clean":"2023-10-06T02:46:28.334936+0000","last_became_active":"2023-10-06T02:46:28.334647+0000","last_became_peered":"2023-10-06T02:46:28.334647+0000","last_unstale":"2023-10-06T02:46:28.334936+0000","last_undegraded":"2023-10-06T02:46:28.334936+0000","last_fullsized":"2023-10-06T02:46:28.334936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:26.287253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.578034+0000","last_change":"2023-10-06T02:46:27.562903+0000","last_active":"2023-10-06T02:46:28.578034+0000","last_peered":"2023-10-06T02:46:28.578034+0000","last_clean":"2023-10-06T02:46:28.578034+0000","last_became_active":"2023-10-06T02:46:27.562623+0000","last_became_peered":"2023-10-06T02:46:27.562623+0000","last_unstale":"2023-10-06T02:46:28.578034+0000","last_undegraded":"2023-10-06T02:46:28.578034+0000","last_fullsized":"2023-10-06T02:46:28.578034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:05.633789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342860+0000","last_change":"2023-10-06T02:46:28.337897+0000","last_active":"2023-10-06T02:46:28.342860+0000","last_peered":"2023-10-06T02:46:28.342860+0000","last_clean":"2023-10-06T02:46:28.342860+0000","last_became_active":"2023-10-06T02:46:28.337513+0000","last_became_peered":"2023-10-06T02:46:28.337513+0000","last_unstale":"2023-10-06T02:46:28.342860+0000","last_undegraded":"2023-10-06T02:46:28.342860+0000","last_fullsized":"2023-10-06T02:46:28.342860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.402447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572555+0000","last_change":"2023-10-06T02:46:27.572135+0000","last_active":"2023-10-06T02:46:28.572555+0000","last_peered":"2023-10-06T02:46:28.572555+0000","last_clean":"2023-10-06T02:46:28.572555+0000","last_became_active":"2023-10-06T02:46:27.570782+0000","last_became_peered":"2023-10-06T02:46:27.570782+0000","last_unstale":"2023-10-06T02:46:28.572555+0000","last_undegraded":"2023-10-06T02:46:28.572555+0000","last_fullsized":"2023-10-06T02:46:28.572555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:43.087819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.336744+0000","last_change":"2023-10-06T02:46:28.336744+0000","last_active":"2023-10-06T02:46:28.336744+0000","last_peered":"2023-10-06T02:46:28.336744+0000","last_clean":"2023-10-06T02:46:28.336744+0000","last_became_active":"2023-10-06T02:46:28.336439+0000","last_became_peered":"2023-10-06T02:46:28.336439+0000","last_unstale":"2023-10-06T02:46:28.336744+0000","last_undegraded":"2023-10-06T02:46:28.336744+0000","last_fullsized":"2023-10-06T02:46:28.336744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:07.794047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572402+0000","last_change":"2023-10-06T02:46:27.566874+0000","last_active":"2023-10-06T02:46:27.572402+0000","last_peered":"2023-10-06T02:46:27.572402+0000","last_clean":"2023-10-06T02:46:27.572402+0000","last_became_active":"2023-10-06T02:46:27.566587+0000","last_became_peered":"2023-10-06T02:46:27.566587+0000","last_unstale":"2023-10-06T02:46:27.572402+0000","last_undegraded":"2023-10-06T02:46:27.572402+0000","last_fullsized":"2023-10-06T02:46:27.572402+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:04.126178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335226+0000","last_change":"2023-10-06T02:46:28.335226+0000","last_active":"2023-10-06T02:46:28.335226+0000","last_peered":"2023-10-06T02:46:28.335226+0000","last_clean":"2023-10-06T02:46:28.335226+0000","last_became_active":"2023-10-06T02:46:28.334935+0000","last_became_peered":"2023-10-06T02:46:28.334935+0000","last_unstale":"2023-10-06T02:46:28.335226+0000","last_undegraded":"2023-10-06T02:46:28.335226+0000","last_fullsized":"2023-10-06T02:46:28.335226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:54.738602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.581409+0000","last_change":"2023-10-06T02:46:27.571174+0000","last_active":"2023-10-06T02:46:27.581409+0000","last_peered":"2023-10-06T02:46:27.581409+0000","last_clean":"2023-10-06T02:46:27.581409+0000","last_became_active":"2023-10-06T02:46:27.570370+0000","last_became_peered":"2023-10-06T02:46:27.570370+0000","last_unstale":"2023-10-06T02:46:27.581409+0000","last_undegraded":"2023-10-06T02:46:27.581409+0000","last_fullsized":"2023-10-06T02:46:27.581409+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:16.553122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.340871+0000","last_change":"2023-10-06T02:46:28.340871+0000","last_active":"2023-10-06T02:46:28.340871+0000","last_peered":"2023-10-06T02:46:28.340871+0000","last_clean":"2023-10-06T02:46:28.340871+0000","last_became_active":"2023-10-06T02:46:28.340360+0000","last_became_peered":"2023-10-06T02:46:28.340360+0000","last_unstale":"2023-10-06T02:46:28.340871+0000","last_undegraded":"2023-10-06T02:46:28.340871+0000","last_fullsized":"2023-10-06T02:46:28.340871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:08.877596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.577845+0000","last_change":"2023-10-06T02:46:27.568740+0000","last_active":"2023-10-06T02:46:28.577845+0000","last_peered":"2023-10-06T02:46:28.577845+0000","last_clean":"2023-10-06T02:46:28.577845+0000","last_became_active":"2023-10-06T02:46:27.568398+0000","last_became_peered":"2023-10-06T02:46:27.568398+0000","last_unstale":"2023-10-06T02:46:28.577845+0000","last_undegraded":"2023-10-06T02:46:28.577845+0000","last_fullsized":"2023-10-06T02:46:28.577845+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:05.364568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335798+0000","last_change":"2023-10-06T02:46:28.331749+0000","last_active":"2023-10-06T02:46:28.335798+0000","last_peered":"2023-10-06T02:46:28.335798+0000","last_clean":"2023-10-06T02:46:28.335798+0000","last_became_active":"2023-10-06T02:46:28.331456+0000","last_became_peered":"2023-10-06T02:46:28.331456+0000","last_unstale":"2023-10-06T02:46:28.335798+0000","last_undegraded":"2023-10-06T02:46:28.335798+0000","last_fullsized":"2023-10-06T02:46:28.335798+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:46:56.307796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.573118+0000","last_change":"2023-10-06T02:46:27.571276+0000","last_active":"2023-10-06T02:46:28.573118+0000","last_peered":"2023-10-06T02:46:28.573118+0000","last_clean":"2023-10-06T02:46:28.573118+0000","last_became_active":"2023-10-06T02:46:27.570942+0000","last_became_peered":"2023-10-06T02:46:27.570942+0000","last_unstale":"2023-10-06T02:46:28.573118+0000","last_undegraded":"2023-10-06T02:46:28.573118+0000","last_fullsized":"2023-10-06T02:46:28.573118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:36.868824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.336910+0000","last_change":"2023-10-06T02:46:28.336910+0000","last_active":"2023-10-06T02:46:28.336910+0000","last_peered":"2023-10-06T02:46:28.336910+0000","last_clean":"2023-10-06T02:46:28.336910+0000","last_became_active":"2023-10-06T02:46:28.336570+0000","last_became_peered":"2023-10-06T02:46:28.336570+0000","last_unstale":"2023-10-06T02:46:28.336910+0000","last_undegraded":"2023-10-06T02:46:28.336910+0000","last_fullsized":"2023-10-06T02:46:28.336910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:23.332330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572813+0000","last_change":"2023-10-06T02:46:27.562648+0000","last_active":"2023-10-06T02:46:28.572813+0000","last_peered":"2023-10-06T02:46:28.572813+0000","last_clean":"2023-10-06T02:46:28.572813+0000","last_became_active":"2023-10-06T02:46:27.562431+0000","last_became_peered":"2023-10-06T02:46:27.562431+0000","last_unstale":"2023-10-06T02:46:28.572813+0000","last_undegraded":"2023-10-06T02:46:28.572813+0000","last_fullsized":"2023-10-06T02:46:28.572813+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:07.865146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342663+0000","last_change":"2023-10-06T02:46:28.334316+0000","last_active":"2023-10-06T02:46:28.342663+0000","last_peered":"2023-10-06T02:46:28.342663+0000","last_clean":"2023-10-06T02:46:28.342663+0000","last_became_active":"2023-10-06T02:46:28.334098+0000","last_became_peered":"2023-10-06T02:46:28.334098+0000","last_unstale":"2023-10-06T02:46:28.342663+0000","last_undegraded":"2023-10-06T02:46:28.342663+0000","last_fullsized":"2023-10-06T02:46:28.342663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:10.268820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.325555+0000","last_change":"2023-10-06T02:46:27.567563+0000","last_active":"2023-10-06T02:46:28.325555+0000","last_peered":"2023-10-06T02:46:28.325555+0000","last_clean":"2023-10-06T02:46:28.325555+0000","last_became_active":"2023-10-06T02:46:27.567238+0000","last_became_peered":"2023-10-06T02:46:27.567238+0000","last_unstale":"2023-10-06T02:46:28.325555+0000","last_undegraded":"2023-10-06T02:46:28.325555+0000","last_fullsized":"2023-10-06T02:46:28.325555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:13:32.322653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342640+0000","last_change":"2023-10-06T02:46:28.333762+0000","last_active":"2023-10-06T02:46:28.342640+0000","last_peered":"2023-10-06T02:46:28.342640+0000","last_clean":"2023-10-06T02:46:28.342640+0000","last_became_active":"2023-10-06T02:46:28.333193+0000","last_became_peered":"2023-10-06T02:46:28.333193+0000","last_unstale":"2023-10-06T02:46:28.342640+0000","last_undegraded":"2023-10-06T02:46:28.342640+0000","last_fullsized":"2023-10-06T02:46:28.342640+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:53.856632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.570935+0000","last_change":"2023-10-06T02:46:27.557781+0000","last_active":"2023-10-06T02:46:27.570935+0000","last_peered":"2023-10-06T02:46:27.570935+0000","last_clean":"2023-10-06T02:46:27.570935+0000","last_became_active":"2023-10-06T02:46:27.557422+0000","last_became_peered":"2023-10-06T02:46:27.557422+0000","last_unstale":"2023-10-06T02:46:27.570935+0000","last_undegraded":"2023-10-06T02:46:27.570935+0000","last_fullsized":"2023-10-06T02:46:27.570935+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:09.891795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334069+0000","last_change":"2023-10-06T02:46:28.334069+0000","last_active":"2023-10-06T02:46:28.334069+0000","last_peered":"2023-10-06T02:46:28.334069+0000","last_clean":"2023-10-06T02:46:28.334069+0000","last_became_active":"2023-10-06T02:46:28.333761+0000","last_became_peered":"2023-10-06T02:46:28.333761+0000","last_unstale":"2023-10-06T02:46:28.334069+0000","last_undegraded":"2023-10-06T02:46:28.334069+0000","last_fullsized":"2023-10-06T02:46:28.334069+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:12.344870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.574306+0000","last_change":"2023-10-06T02:46:27.570952+0000","last_active":"2023-10-06T02:46:28.574306+0000","last_peered":"2023-10-06T02:46:28.574306+0000","last_clean":"2023-10-06T02:46:28.574306+0000","last_became_active":"2023-10-06T02:46:27.567517+0000","last_became_peered":"2023-10-06T02:46:27.567517+0000","last_unstale":"2023-10-06T02:46:28.574306+0000","last_undegraded":"2023-10-06T02:46:28.574306+0000","last_fullsized":"2023-10-06T02:46:28.574306+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:58.977642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.328600+0000","last_change":"2023-10-06T02:46:28.328600+0000","last_active":"2023-10-06T02:46:28.328600+0000","last_peered":"2023-10-06T02:46:28.328600+0000","last_clean":"2023-10-06T02:46:28.328600+0000","last_became_active":"2023-10-06T02:46:28.328160+0000","last_became_peered":"2023-10-06T02:46:28.328160+0000","last_unstale":"2023-10-06T02:46:28.328600+0000","last_undegraded":"2023-10-06T02:46:28.328600+0000","last_fullsized":"2023-10-06T02:46:28.328600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:01.983303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.571051+0000","last_change":"2023-10-06T02:46:27.568939+0000","last_active":"2023-10-06T02:46:27.571051+0000","last_peered":"2023-10-06T02:46:27.571051+0000","last_clean":"2023-10-06T02:46:27.571051+0000","last_became_active":"2023-10-06T02:46:27.568625+0000","last_became_peered":"2023-10-06T02:46:27.568625+0000","last_unstale":"2023-10-06T02:46:27.571051+0000","last_undegraded":"2023-10-06T02:46:27.571051+0000","last_fullsized":"2023-10-06T02:46:27.571051+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:00.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.340639+0000","last_change":"2023-10-06T02:46:28.336965+0000","last_active":"2023-10-06T02:46:28.340639+0000","last_peered":"2023-10-06T02:46:28.340639+0000","last_clean":"2023-10-06T02:46:28.340639+0000","last_became_active":"2023-10-06T02:46:28.336452+0000","last_became_peered":"2023-10-06T02:46:28.336452+0000","last_unstale":"2023-10-06T02:46:28.340639+0000","last_undegraded":"2023-10-06T02:46:28.340639+0000","last_fullsized":"2023-10-06T02:46:28.340639+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:36.444179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574710+0000","last_change":"2023-10-06T02:46:27.568309+0000","last_active":"2023-10-06T02:46:27.574710+0000","last_peered":"2023-10-06T02:46:27.574710+0000","last_clean":"2023-10-06T02:46:27.574710+0000","last_became_active":"2023-10-06T02:46:27.567845+0000","last_became_peered":"2023-10-06T02:46:27.567845+0000","last_unstale":"2023-10-06T02:46:27.574710+0000","last_undegraded":"2023-10-06T02:46:27.574710+0000","last_fullsized":"2023-10-06T02:46:27.574710+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:02.210846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.347154+0000","last_change":"2023-10-06T02:46:28.347154+0000","last_active":"2023-10-06T02:46:28.347154+0000","last_peered":"2023-10-06T02:46:28.347154+0000","last_clean":"2023-10-06T02:46:28.347154+0000","last_became_active":"2023-10-06T02:46:28.344633+0000","last_became_peered":"2023-10-06T02:46:28.344633+0000","last_unstale":"2023-10-06T02:46:28.347154+0000","last_undegraded":"2023-10-06T02:46:28.347154+0000","last_fullsized":"2023-10-06T02:46:28.347154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:59.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572523+0000","last_change":"2023-10-06T02:46:27.568499+0000","last_active":"2023-10-06T02:46:27.572523+0000","last_peered":"2023-10-06T02:46:27.572523+0000","last_clean":"2023-10-06T02:46:27.572523+0000","last_became_active":"2023-10-06T02:46:27.568188+0000","last_became_peered":"2023-10-06T02:46:27.568188+0000","last_unstale":"2023-10-06T02:46:27.572523+0000","last_undegraded":"2023-10-06T02:46:27.572523+0000","last_fullsized":"2023-10-06T02:46:27.572523+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:48.797840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.334289+0000","last_change":"2023-10-06T02:46:28.334289+0000","last_active":"2023-10-06T02:46:28.334289+0000","last_peered":"2023-10-06T02:46:28.334289+0000","last_clean":"2023-10-06T02:46:28.334289+0000","last_became_active":"2023-10-06T02:46:28.333373+0000","last_became_peered":"2023-10-06T02:46:28.333373+0000","last_unstale":"2023-10-06T02:46:28.334289+0000","last_undegraded":"2023-10-06T02:46:28.334289+0000","last_fullsized":"2023-10-06T02:46:28.334289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:51.403042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572421+0000","last_change":"2023-10-06T02:46:27.566922+0000","last_active":"2023-10-06T02:46:27.572421+0000","last_peered":"2023-10-06T02:46:27.572421+0000","last_clean":"2023-10-06T02:46:27.572421+0000","last_became_active":"2023-10-06T02:46:27.566623+0000","last_became_peered":"2023-10-06T02:46:27.566623+0000","last_unstale":"2023-10-06T02:46:27.572421+0000","last_undegraded":"2023-10-06T02:46:27.572421+0000","last_fullsized":"2023-10-06T02:46:27.572421+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:09.343776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.322478+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:46:28.322478+0000","last_peered":"2023-10-06T02:46:28.322478+0000","last_clean":"2023-10-06T02:46:28.322478+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:46:28.322478+0000","last_undegraded":"2023-10-06T02:46:28.322478+0000","last_fullsized":"2023-10-06T02:46:28.322478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339073+0000","last_change":"2023-10-06T02:46:28.339073+0000","last_active":"2023-10-06T02:46:28.339073+0000","last_peered":"2023-10-06T02:46:28.339073+0000","last_clean":"2023-10-06T02:46:28.339073+0000","last_became_active":"2023-10-06T02:46:28.338463+0000","last_became_peered":"2023-10-06T02:46:28.338463+0000","last_unstale":"2023-10-06T02:46:28.339073+0000","last_undegraded":"2023-10-06T02:46:28.339073+0000","last_fullsized":"2023-10-06T02:46:28.339073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:13.216359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.320350+0000","last_change":"2023-10-06T02:46:27.571325+0000","last_active":"2023-10-06T02:46:28.320350+0000","last_peered":"2023-10-06T02:46:28.320350+0000","last_clean":"2023-10-06T02:46:28.320350+0000","last_became_active":"2023-10-06T02:46:27.570938+0000","last_became_peered":"2023-10-06T02:46:27.570938+0000","last_unstale":"2023-10-06T02:46:28.320350+0000","last_undegraded":"2023-10-06T02:46:28.320350+0000","last_fullsized":"2023-10-06T02:46:28.320350+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:26.518332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.554220+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:46:27.554220+0000","last_peered":"2023-10-06T02:46:27.554220+0000","last_clean":"2023-10-06T02:46:27.554220+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:46:27.554220+0000","last_undegraded":"2023-10-06T02:46:27.554220+0000","last_fullsized":"2023-10-06T02:46:27.554220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.335646+0000","last_change":"2023-10-06T02:46:28.335646+0000","last_active":"2023-10-06T02:46:28.335646+0000","last_peered":"2023-10-06T02:46:28.335646+0000","last_clean":"2023-10-06T02:46:28.335646+0000","last_became_active":"2023-10-06T02:46:28.335354+0000","last_became_peered":"2023-10-06T02:46:28.335354+0000","last_unstale":"2023-10-06T02:46:28.335646+0000","last_undegraded":"2023-10-06T02:46:28.335646+0000","last_fullsized":"2023-10-06T02:46:28.335646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:06.262037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.569268+0000","last_change":"2023-10-06T02:46:27.566156+0000","last_active":"2023-10-06T02:46:27.569268+0000","last_peered":"2023-10-06T02:46:27.569268+0000","last_clean":"2023-10-06T02:46:27.569268+0000","last_became_active":"2023-10-06T02:46:27.565209+0000","last_became_peered":"2023-10-06T02:46:27.565209+0000","last_unstale":"2023-10-06T02:46:27.569268+0000","last_undegraded":"2023-10-06T02:46:27.569268+0000","last_fullsized":"2023-10-06T02:46:27.569268+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:02.210846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.556904+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:46:27.556904+0000","last_peered":"2023-10-06T02:46:27.556904+0000","last_clean":"2023-10-06T02:46:27.556904+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:46:27.556904+0000","last_undegraded":"2023-10-06T02:46:27.556904+0000","last_fullsized":"2023-10-06T02:46:27.556904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.339562+0000","last_change":"2023-10-06T02:46:28.336152+0000","last_active":"2023-10-06T02:46:28.339562+0000","last_peered":"2023-10-06T02:46:28.339562+0000","last_clean":"2023-10-06T02:46:28.339562+0000","last_became_active":"2023-10-06T02:46:28.335881+0000","last_became_peered":"2023-10-06T02:46:28.335881+0000","last_unstale":"2023-10-06T02:46:28.339562+0000","last_undegraded":"2023-10-06T02:46:28.339562+0000","last_fullsized":"2023-10-06T02:46:28.339562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:32:32.456718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.581316+0000","last_change":"2023-10-06T02:46:27.575418+0000","last_active":"2023-10-06T02:46:27.581316+0000","last_peered":"2023-10-06T02:46:27.581316+0000","last_clean":"2023-10-06T02:46:27.581316+0000","last_became_active":"2023-10-06T02:46:27.575032+0000","last_became_peered":"2023-10-06T02:46:27.575032+0000","last_unstale":"2023-10-06T02:46:27.581316+0000","last_undegraded":"2023-10-06T02:46:27.581316+0000","last_fullsized":"2023-10-06T02:46:27.581316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:09.343776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.556618+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:46:27.556618+0000","last_peered":"2023-10-06T02:46:27.556618+0000","last_clean":"2023-10-06T02:46:27.556618+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:46:27.556618+0000","last_undegraded":"2023-10-06T02:46:27.556618+0000","last_fullsized":"2023-10-06T02:46:27.556618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.342849+0000","last_change":"2023-10-06T02:46:28.337789+0000","last_active":"2023-10-06T02:46:28.342849+0000","last_peered":"2023-10-06T02:46:28.342849+0000","last_clean":"2023-10-06T02:46:28.342849+0000","last_became_active":"2023-10-06T02:46:28.337484+0000","last_became_peered":"2023-10-06T02:46:28.337484+0000","last_unstale":"2023-10-06T02:46:28.342849+0000","last_undegraded":"2023-10-06T02:46:28.342849+0000","last_fullsized":"2023-10-06T02:46:28.342849+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:49.648521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574895+0000","last_change":"2023-10-06T02:46:27.566684+0000","last_active":"2023-10-06T02:46:27.574895+0000","last_peered":"2023-10-06T02:46:27.574895+0000","last_clean":"2023-10-06T02:46:27.574895+0000","last_became_active":"2023-10-06T02:46:27.564717+0000","last_became_peered":"2023-10-06T02:46:27.564717+0000","last_unstale":"2023-10-06T02:46:27.574895+0000","last_undegraded":"2023-10-06T02:46:27.574895+0000","last_fullsized":"2023-10-06T02:46:27.574895+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:25.133829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.564012+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:46:27.564012+0000","last_peered":"2023-10-06T02:46:27.564012+0000","last_clean":"2023-10-06T02:46:27.564012+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:46:27.564012+0000","last_undegraded":"2023-10-06T02:46:27.564012+0000","last_fullsized":"2023-10-06T02:46:27.564012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.333207+0000","last_change":"2023-10-06T02:46:28.333207+0000","last_active":"2023-10-06T02:46:28.333207+0000","last_peered":"2023-10-06T02:46:28.333207+0000","last_clean":"2023-10-06T02:46:28.333207+0000","last_became_active":"2023-10-06T02:46:28.332879+0000","last_became_peered":"2023-10-06T02:46:28.332879+0000","last_unstale":"2023-10-06T02:46:28.333207+0000","last_undegraded":"2023-10-06T02:46:28.333207+0000","last_fullsized":"2023-10-06T02:46:28.333207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:54.979203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.572545+0000","last_change":"2023-10-06T02:46:27.568852+0000","last_active":"2023-10-06T02:46:27.572545+0000","last_peered":"2023-10-06T02:46:27.572545+0000","last_clean":"2023-10-06T02:46:27.572545+0000","last_became_active":"2023-10-06T02:46:27.568517+0000","last_became_peered":"2023-10-06T02:46:27.568517+0000","last_unstale":"2023-10-06T02:46:27.572545+0000","last_undegraded":"2023-10-06T02:46:27.572545+0000","last_fullsized":"2023-10-06T02:46:27.572545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:38.294381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.324472+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:46:28.324472+0000","last_peered":"2023-10-06T02:46:28.324472+0000","last_clean":"2023-10-06T02:46:28.324472+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:46:28.324472+0000","last_undegraded":"2023-10-06T02:46:28.324472+0000","last_fullsized":"2023-10-06T02:46:28.324472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.338652+0000","last_change":"2023-10-06T02:46:28.338652+0000","last_active":"2023-10-06T02:46:28.338652+0000","last_peered":"2023-10-06T02:46:28.338652+0000","last_clean":"2023-10-06T02:46:28.338652+0000","last_became_active":"2023-10-06T02:46:28.338321+0000","last_became_peered":"2023-10-06T02:46:28.338321+0000","last_unstale":"2023-10-06T02:46:28.338652+0000","last_undegraded":"2023-10-06T02:46:28.338652+0000","last_fullsized":"2023-10-06T02:46:28.338652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:54.979203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.574667+0000","last_change":"2023-10-06T02:46:27.568158+0000","last_active":"2023-10-06T02:46:27.574667+0000","last_peered":"2023-10-06T02:46:27.574667+0000","last_clean":"2023-10-06T02:46:27.574667+0000","last_became_active":"2023-10-06T02:46:27.567886+0000","last_became_peered":"2023-10-06T02:46:27.567886+0000","last_unstale":"2023-10-06T02:46:27.574667+0000","last_undegraded":"2023-10-06T02:46:27.574667+0000","last_fullsized":"2023-10-06T02:46:27.574667+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:05.293970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.552721+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:46:27.552721+0000","last_peered":"2023-10-06T02:46:27.552721+0000","last_clean":"2023-10-06T02:46:27.552721+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:46:27.552721+0000","last_undegraded":"2023-10-06T02:46:27.552721+0000","last_fullsized":"2023-10-06T02:46:27.552721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.344918+0000","last_change":"2023-10-06T02:46:28.344918+0000","last_active":"2023-10-06T02:46:28.344918+0000","last_peered":"2023-10-06T02:46:28.344918+0000","last_clean":"2023-10-06T02:46:28.344918+0000","last_became_active":"2023-10-06T02:46:28.344500+0000","last_became_peered":"2023-10-06T02:46:28.344500+0000","last_unstale":"2023-10-06T02:46:28.344918+0000","last_undegraded":"2023-10-06T02:46:28.344918+0000","last_fullsized":"2023-10-06T02:46:28.344918+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:27.530957+0000","last_clean_scrub_stamp":"2023-10-06T02:46:27.530957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:08.877596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.572822+0000","last_change":"2023-10-06T02:46:27.574660+0000","last_active":"2023-10-06T02:46:28.572822+0000","last_peered":"2023-10-06T02:46:28.572822+0000","last_clean":"2023-10-06T02:46:28.572822+0000","last_became_active":"2023-10-06T02:46:27.574391+0000","last_became_peered":"2023-10-06T02:46:27.574391+0000","last_unstale":"2023-10-06T02:46:28.572822+0000","last_undegraded":"2023-10-06T02:46:28.572822+0000","last_fullsized":"2023-10-06T02:46:28.572822+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:46:26.518507+0000","last_clean_scrub_stamp":"2023-10-06T02:46:26.518507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:20.105815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.563846+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:46:27.563846+0000","last_peered":"2023-10-06T02:46:27.563846+0000","last_clean":"2023-10-06T02:46:27.563846+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:46:27.563846+0000","last_undegraded":"2023-10-06T02:46:27.563846+0000","last_fullsized":"2023-10-06T02:46:27.563846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T02:46:27.555148+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:46:27.555148+0000","last_peered":"2023-10-06T02:46:27.555148+0000","last_clean":"2023-10-06T02:46:27.555148+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:46:27.555148+0000","last_undegraded":"2023-10-06T02:46:27.555148+0000","last_fullsized":"2023-10-06T02:46:27.555148+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":810542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.105}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.018},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.92300000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.073}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":503808,"data_stored":220728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.98599999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.0960000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.038}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.155},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0740000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.224}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297776,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074064,"statfs":{"total":96636764160,"available":96331841536,"internally_reserved":0,"allocated":499712,"data_stored":220154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.117}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.1830000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.97299999999999998}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.304},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.022}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.2909999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.4670000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":221620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.107},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.056}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.996},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.94499999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.282},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.246}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.3779999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.429},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.3320000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297784,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074056,"statfs":{"total":96636764160,"available":96331833344,"internally_reserved":0,"allocated":507904,"data_stored":220694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0289999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.874},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.663},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.748}]}]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":221642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81299999999999994}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.012}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.5600000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.331},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1140000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.5029999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297776,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074064,"statfs":{"total":96636764160,"available":96331841536,"internally_reserved":0,"allocated":499712,"data_stored":220154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.751}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.629},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1850000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.274},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0700000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.3140000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.147},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1140000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":810486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.05}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.2849999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":1.2070000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:46:31.907 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T02:46:31.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T02:46:32.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 Processor -- start 2023-10-06T02:46:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 -- start start 2023-10-06T02:46:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb3581589f0 0x7fb358158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3580af3a0 0x7fb3580af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3580cabb0 con 0x7fb3580685b0 2023-10-06T02:46:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb358159300 con 0x7fb3580af3a0 2023-10-06T02:46:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.025+0000 7fb35fc2c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3580afe20 con 0x7fb3581589f0 2023-10-06T02:46:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d1c7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3580af3a0 0x7fb3580af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35e1c9700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb3580685b0 0x7fb35815ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60788/0 (socket says 172.21.15.33:60788) 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35e1c9700 1 -- 172.21.15.33:0/3423781112 learned_addr learned my addr 172.21.15.33:0/3423781112 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d9c8700 1 --2- 172.21.15.33:0/3423781112 >> v2:172.21.15.167:3300/0 conn(0x7fb3581589f0 0x7fb358158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3423781112 >> v2:172.21.15.167:3300/0 conn(0x7fb3581589f0 msgr2=0x7fb358158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d1c7700 1 --2- 172.21.15.33:0/3423781112 >> v2:172.21.15.167:3300/0 conn(0x7fb3581589f0 0x7fb358158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3423781112 >> v1:172.21.15.33:6789/0 conn(0x7fb3580685b0 legacy=0x7fb35815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35c9c6700 1 -- 172.21.15.33:0/3423781112 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017680300 0 0) 0x7fb3580cabb0 con 0x7fb3580685b0 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3423781112 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580b0100 con 0x7fb3580af3a0 2023-10-06T02:46:32.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35d1c7700 1 --2- 172.21.15.33:0/3423781112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3580af7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb358064100 tx=0x7fb3540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4535637455c82d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35c9c6700 1 -- 172.21.15.33:0/3423781112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35400b6b0 con 0x7fb3580af3a0 2023-10-06T02:46:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35c9c6700 1 -- 172.21.15.33:0/3423781112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35400b810 con 0x7fb3580af3a0 2023-10-06T02:46:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35c9c6700 1 -- 172.21.15.33:0/3423781112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35400bbb0 con 0x7fb3580af3a0 2023-10-06T02:46:32.034 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3423781112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 msgr2=0x7fb3580af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3423781112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3580af7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb358064100 tx=0x7fb3540091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3423781112 shutdown_connections 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3423781112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3580af7b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3423781112 >> v2:172.21.15.167:3300/0 conn(0x7fb3581589f0 0x7fb358158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3423781112 >> 172.21.15.33:0/3423781112 conn(0x7fb358066fb0 msgr2=0x7fb3580673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3423781112 shutdown_connections 2023-10-06T02:46:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.029+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3423781112 wait complete. 2023-10-06T02:46:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 Processor -- start 2023-10-06T02:46:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 -- start start 2023-10-06T02:46:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3580685b0 0x7fb358154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3581594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3581589f0 0x7fb35815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815a930 con 0x7fb3580af3a0 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb358159d20 con 0x7fb3581589f0 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35fc2c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815a000 con 0x7fb3580685b0 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3581594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3581594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36108/0 (socket says 172.21.15.33:36108) 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3654965462 learned_addr learned my addr 172.21.15.33:0/3654965462 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d9c8700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3580685b0 0x7fb358154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35e1c9700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3581589f0 0x7fb35815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3580685b0 msgr2=0x7fb358154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3580685b0 0x7fb358154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3581589f0 msgr2=0x7fb35815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3581589f0 0x7fb35815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 -- 172.21.15.33:0/3654965462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb354009420 con 0x7fb3580af3a0 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d9c8700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3580685b0 0x7fb358154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35e1c9700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3581589f0 0x7fb35815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb35d1c7700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3581594d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb358087b00 tx=0x7fb354017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.033+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354011420 con 0x7fb3580af3a0 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb354011580 con 0x7fb3580af3a0 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35815a2e0 con 0x7fb3580af3a0 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354011940 con 0x7fb3580af3a0 2023-10-06T02:46:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3581548a0 con 0x7fb3580af3a0 2023-10-06T02:46:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35815aaa0 con 0x7fb3580af3a0 2023-10-06T02:46:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb354011d30 con 0x7fb3580af3a0 2023-10-06T02:46:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb34dffb700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb33c060550 0x7fb33c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb35407dfe0 con 0x7fb3580af3a0 2023-10-06T02:46:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.037+0000 7fb35d9c8700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb33c060550 0x7fb33c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.041+0000 7fb35d9c8700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb33c060550 0x7fb33c062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb3581583a0 tx=0x7fb348008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.049+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb35404e2b0 con 0x7fb3580af3a0 2023-10-06T02:46:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.269+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb3580710b0 con 0x7fb3580af3a0 2023-10-06T02:46:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.277+0000 7fb34dffb700 1 -- 172.21.15.33:0/3654965462 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb3540150e0 con 0x7fb3580af3a0 2023-10-06T02:46:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.277+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb33c060550 msgr2=0x7fb33c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.277+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb33c060550 0x7fb33c062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb3581583a0 tx=0x7fb348008040 comp rx=0 tx=0).stop 2023-10-06T02:46:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.277+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 msgr2=0x7fb3581594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.277+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3581594d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb358087b00 tx=0x7fb354017000 comp rx=0 tx=0).stop 2023-10-06T02:46:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 shutdown_connections 2023-10-06T02:46:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3581589f0 0x7fb35815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3580af3a0 0x7fb3581594d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb33c060550 0x7fb33c062a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 --2- 172.21.15.33:0/3654965462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3580685b0 0x7fb358154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 >> 172.21.15.33:0/3654965462 conn(0x7fb358066fb0 msgr2=0x7fb3580b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 shutdown_connections 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.281+0000 7fb35fc2c700 1 -- 172.21.15.33:0/3654965462 wait complete. 2023-10-06T02:46:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2023-10-06T02:46:32.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:32.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 Processor -- start 2023-10-06T02:46:32.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- start start 2023-10-06T02:46:32.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb94815a010 0x7fb9480cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb94815a3e0 0x7fb9480cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9480cce00 con 0x7fb9480cbc70 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480cd7d0 con 0x7fb94815a010 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480cc1f0 con 0x7fb94815a3e0 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb94815a010 0x7fb9480cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb94815a010 0x7fb9480cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36124/0 (socket says 172.21.15.33:36124) 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 -- 172.21.15.33:0/2565589188 learned_addr learned my addr 172.21.15.33:0/2565589188 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:32.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb9467fc700 1 --2- 172.21.15.33:0/2565589188 >> v2:172.21.15.167:3300/0 conn(0x7fb94815a3e0 0x7fb9480cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 -- 172.21.15.33:0/2565589188 >> v2:172.21.15.167:3300/0 conn(0x7fb94815a3e0 msgr2=0x7fb9480cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 --2- 172.21.15.33:0/2565589188 >> v2:172.21.15.167:3300/0 conn(0x7fb94815a3e0 0x7fb9480cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 -- 172.21.15.33:0/2565589188 >> v1:172.21.15.167:6789/0 conn(0x7fb9480cbc70 legacy=0x7fb94806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 -- 172.21.15.33:0/2565589188 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480cc4d0 con 0x7fb94815a010 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb946ffd700 1 --2- 172.21.15.33:0/2565589188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480cd290 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb94806e050 tx=0x7fb930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821f5f49aad678c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb945ffb700 1 -- 172.21.15.33:0/2565589188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000b860 con 0x7fb94815a010 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb945ffb700 1 -- 172.21.15.33:0/2565589188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93000b9c0 con 0x7fb94815a010 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb945ffb700 1 -- 172.21.15.33:0/2565589188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000bd60 con 0x7fb94815a010 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- 172.21.15.33:0/2565589188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 msgr2=0x7fb9480cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/2565589188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480cd290 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb94806e050 tx=0x7fb930009250 comp rx=0 tx=0).stop 2023-10-06T02:46:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- 172.21.15.33:0/2565589188 shutdown_connections 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/2565589188 >> v2:172.21.15.167:3300/0 conn(0x7fb94815a3e0 0x7fb9480cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/2565589188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480cd290 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- 172.21.15.33:0/2565589188 >> 172.21.15.33:0/2565589188 conn(0x7fb94806c450 msgr2=0x7fb9480cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- 172.21.15.33:0/2565589188 shutdown_connections 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.405+0000 7fb94d9e5700 1 -- 172.21.15.33:0/2565589188 wait complete. 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 Processor -- start 2023-10-06T02:46:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 -- start start 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 0x7fb9480c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9480c7780 0x7fb9480ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814e180 con 0x7fb94815a010 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480b0900 con 0x7fb9480c7780 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb94d9e5700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480b0be0 con 0x7fb9480cbc70 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb9467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb946ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 0x7fb9480c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb9477fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9480c7780 0x7fb9480ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb946ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 0x7fb9480c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:41570/0 (socket says 172.21.15.33:41570) 2023-10-06T02:46:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb9467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36140/0 (socket says 172.21.15.33:36140) 2023-10-06T02:46:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb946ffd700 1 -- 172.21.15.33:0/1470497693 learned_addr learned my addr 172.21.15.33:0/1470497693 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb9467fc700 1 -- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 msgr2=0x7fb9480c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.409+0000 7fb9467fc700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 0x7fb9480c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb9467fc700 1 -- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9480c7780 msgr2=0x7fb9480ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb9467fc700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9480c7780 0x7fb9480ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb9467fc700 1 -- 172.21.15.33:0/1470497693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300094f0 con 0x7fb94815a010 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb946ffd700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 0x7fb9480c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb9477fe700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9480c7780 0x7fb9480ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb9467fc700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480b22e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9480c8130 tx=0x7fb938008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:32.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938012630 con 0x7fb94815a010 2023-10-06T02:46:32.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938012790 con 0x7fb94815a010 2023-10-06T02:46:32.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9480b2be0 con 0x7fb94815a010 2023-10-06T02:46:32.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938012b30 con 0x7fb94815a010 2023-10-06T02:46:32.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9480b31a0 con 0x7fb94815a010 2023-10-06T02:46:32.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9480b0d90 con 0x7fb94815a010 2023-10-06T02:46:32.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.413+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb93801d400 con 0x7fb94815a010 2023-10-06T02:46:32.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.417+0000 7fb93effd700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb92c060550 0x7fb92c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:32.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.417+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb93807cc10 con 0x7fb94815a010 2023-10-06T02:46:32.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.417+0000 7fb946ffd700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb92c060550 0x7fb92c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:32.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.417+0000 7fb946ffd700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb92c060550 0x7fb92c062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb93000c010 tx=0x7fb930017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.421+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb9480b0d90 con 0x7fb94815a010 2023-10-06T02:46:32.623 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.617+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9480b0d50 con 0x7fb94815a010 2023-10-06T02:46:32.624 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb93effd700 1 -- 172.21.15.33:0/1470497693 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fb93804cf40 con 0x7fb94815a010 2023-10-06T02:46:32.624 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:32.624 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb92c060550 msgr2=0x7fb92c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb92c060550 0x7fb92c062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb93000c010 tx=0x7fb930017000 comp rx=0 tx=0).stop 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 msgr2=0x7fb9480b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480b22e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9480c8130 tx=0x7fb938008040 comp rx=0 tx=0).stop 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 shutdown_connections 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9480c7780 0x7fb9480ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb94815a010 0x7fb9480b22e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb92c060550 0x7fb92c062a00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 --2- 172.21.15.33:0/1470497693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9480cbc70 0x7fb9480c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 >> 172.21.15.33:0/1470497693 conn(0x7fb94806c450 msgr2=0x7fb94814e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 shutdown_connections 2023-10-06T02:46:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:32.621+0000 7fb94d9e5700 1 -- 172.21.15.33:0/1470497693 wait complete. 2023-10-06T02:46:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:33.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T02:46:27.926852+0000', 'modified': '2023-10-06T02:46:30.566783+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5145}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5145': {'gid': 5145, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/3594813995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 3594813995}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 3594813995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T02:46:33.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T02:46:33.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T02:46:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.157+0000 7f36f7fff700 1 Processor -- start 2023-10-06T02:46:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.157+0000 7f36f7fff700 1 -- start start 2023-10-06T02:46:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f815ace0 0x7f36f8161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f8157eb0 con 0x7f36f80adb60 2023-10-06T02:46:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f8069820 con 0x7f36f8067970 2023-10-06T02:46:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8067ea0 con 0x7f36f815ace0 2023-10-06T02:46:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f6ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f36f80adb60 0x7f36f80c6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45468/0 (socket says 172.21.15.33:45468) 2023-10-06T02:46:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f6ffd700 1 -- 172.21.15.33:0/94983254 learned_addr learned my addr 172.21.15.33:0/94983254 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f77fe700 1 --2- 172.21.15.33:0/94983254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f815ace0 0x7f36f8161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142579607 0 0) 0x7f36e8001110 con 0x7f36f80adb60 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d8003570 con 0x7f36f80adb60 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615260659 0 0) 0x7f36f8069820 con 0x7f36f8067970 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e8001110 con 0x7f36f8067970 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f77fe700 1 -- 172.21.15.33:0/94983254 >> v1:172.21.15.167:6789/0 conn(0x7f36f8067970 legacy=0x7f36f815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f77fe700 1 -- 172.21.15.33:0/94983254 >> v1:172.21.15.33:6789/0 conn(0x7f36f80adb60 legacy=0x7f36f80c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f77fe700 1 -- 172.21.15.33:0/94983254 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f8068180 con 0x7f36f815ace0 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f77fe700 1 --2- 172.21.15.33:0/94983254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f815ace0 0x7f36f8161970 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e90 tx=0x7f36f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8bff8370385a5c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000e070 con 0x7f36f815ace0 2023-10-06T02:46:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f000bb80 con 0x7f36f815ace0 2023-10-06T02:46:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f5ffb700 1 -- 172.21.15.33:0/94983254 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0012430 con 0x7f36f815ace0 2023-10-06T02:46:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- 172.21.15.33:0/94983254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f815ace0 msgr2=0x7f36f8161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 --2- 172.21.15.33:0/94983254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f815ace0 0x7f36f8161970 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009e90 tx=0x7f36f0009780 comp rx=0 tx=0).stop 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- 172.21.15.33:0/94983254 shutdown_connections 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 --2- 172.21.15.33:0/94983254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f815ace0 0x7f36f8161970 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- 172.21.15.33:0/94983254 >> 172.21.15.33:0/94983254 conn(0x7f36f80c8240 msgr2=0x7f36f80c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- 172.21.15.33:0/94983254 shutdown_connections 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.161+0000 7f36f7fff700 1 -- 172.21.15.33:0/94983254 wait complete. 2023-10-06T02:46:33.168 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 Processor -- start 2023-10-06T02:46:33.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 -- start start 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f8067970 0x7f36f8167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 0x7f36f81696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f815ace0 0x7f36f8166890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8169c30 con 0x7f36f80adb60 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8167190 con 0x7f36f815ace0 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f8167470 con 0x7f36f8067970 2023-10-06T02:46:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 0x7f36f81696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 0x7f36f81696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41194/0 (socket says 172.21.15.33:41194) 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 -- 172.21.15.33:0/3797747302 learned_addr learned my addr 172.21.15.33:0/3797747302 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f815ace0 0x7f36f8166890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f8067970 0x7f36f8167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 -- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f8067970 msgr2=0x7f36f8167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f8067970 0x7f36f8167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 -- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f815ace0 msgr2=0x7f36f8166890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f815ace0 0x7f36f8166890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 -- 172.21.15.33:0/3797747302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0009a20 con 0x7f36f80adb60 2023-10-06T02:46:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f815ace0 0x7f36f8166890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 0x7f36f81696f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c790 tx=0x7f36e0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36eeffd700 1 -- 172.21.15.33:0/3797747302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0010510 con 0x7f36f80adb60 2023-10-06T02:46:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36eeffd700 1 -- 172.21.15.33:0/3797747302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0010670 con 0x7f36f80adb60 2023-10-06T02:46:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.165+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f8167750 con 0x7f36f80adb60 2023-10-06T02:46:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.169+0000 7f36eeffd700 1 -- 172.21.15.33:0/3797747302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0010a10 con 0x7f36f80adb60 2023-10-06T02:46:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.169+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f80a47a0 con 0x7f36f80adb60 2023-10-06T02:46:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.169+0000 7f36eeffd700 1 -- 172.21.15.33:0/3797747302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f36e001b400 con 0x7f36f80adb60 2023-10-06T02:46:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.169+0000 7f36eeffd700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f36dc060550 0x7f36dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.169+0000 7f36eeffd700 1 -- 172.21.15.33:0/3797747302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f36e007c920 con 0x7f36f80adb60 2023-10-06T02:46:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f36dc060550 0x7f36dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f36dc060550 0x7f36dc062a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36f80b0da0 tx=0x7f36e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36ff891740 1 Processor -- start 2023-10-06T02:46:33.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36ff891740 1 -- start start 2023-10-06T02:46:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36ff891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1a6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a6d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.173+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1a6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x199af30 con 0x17efae0 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x1996ed0 con 0x183a790 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x195d0b0 con 0x17ef710 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1a6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1a6ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41202/0 (socket says 172.21.15.33:41202) 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 -- 172.21.15.33:0/2938371541 learned_addr learned my addr 172.21.15.33:0/2938371541 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f67fc700 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a6d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f77fe700 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1a6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 -- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 msgr2=0x1a6d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a6d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 -- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 msgr2=0x1a6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1a6d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 -- 172.21.15.33:0/2938371541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x183b9e0 con 0x17efae0 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1a6ce50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f36e8005d10 tx=0x7f36e8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9b2aacb7b45e25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36e7fff700 1 -- 172.21.15.33:0/2938371541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8005900 con 0x17efae0 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36e7fff700 1 -- 172.21.15.33:0/2938371541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e800e630 con 0x17efae0 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36e7fff700 1 -- 172.21.15.33:0/2938371541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800e9f0 con 0x17efae0 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- 172.21.15.33:0/2938371541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 msgr2=0x1a6ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1a6ce50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f36e8005d10 tx=0x7f36e8013000 comp rx=0 tx=0).stop 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- 172.21.15.33:0/2938371541 shutdown_connections 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a6d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1a6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 --2- 172.21.15.33:0/2938371541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1a6ce50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- 172.21.15.33:0/2938371541 >> 172.21.15.33:0/2938371541 conn(0x17efeb0 msgr2=0x1a29880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- 172.21.15.33:0/2938371541 shutdown_connections 2023-10-06T02:46:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.177+0000 7f36ff891740 1 -- 172.21.15.33:0/2938371541 wait complete. 2023-10-06T02:46:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 -- start start 2023-10-06T02:46:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 Processor -- start 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 0x1a52c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x183a790 0x1a53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x195cd50 con 0x17ef710 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x1935970 con 0x17efae0 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x19966e0 con 0x183a790 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 0x1a52c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36ff891740 10 client.0 _ll_register_callbacks cb 0x19e0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:46:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 0x1a52c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41208/0 (socket says 172.21.15.33:41208) 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3600866294 learned_addr learned my addr 172.21.15.33:0/3600866294 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x183a790 0x1a53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x183a790 msgr2=0x1a53680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x183a790 0x1a53680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 msgr2=0x1a53140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a53140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3600866294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e800f040 con 0x17ef710 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x183a790 0x1a53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 0x1a52c00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x1939000 tx=0x7f36e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f2ab34cfc2fc1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.181+0000 7f36e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T02:46:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 1 -- 172.21.15.33:0/3600866294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8028070 con 0x17ef710 2023-10-06T02:46:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 1 -- 172.21.15.33:0/3600866294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f36e800ee70 con 0x17ef710 2023-10-06T02:46:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1935cc0 con 0x17ef710 2023-10-06T02:46:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 1 -- 172.21.15.33:0/3600866294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36e800e9b0 con 0x17ef710 2023-10-06T02:46:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 10 client.5340 fetch_fsmap learned FSMap version 16 2023-10-06T02:46:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x1965370 con 0x17ef710 2023-10-06T02:46:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 1 -- 172.21.15.33:0/3600866294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1935cc0 con 0x17ef710 2023-10-06T02:46:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 1 -- 172.21.15.33:0/3600866294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f36e802d460 con 0x17ef710 2023-10-06T02:46:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 10 client.5340 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T02:46:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 10 client.5340 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:46:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 20 client.5340 populate_metadata read hostname 'smithi033' 2023-10-06T02:46:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1a798c0 0x1a7bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 4 client.5340 mds_command: new command op to 5145 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 --> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x19393e0 con 0x1a798c0 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1a798c0 0x1a7bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 10 client.5340 ms_handle_connect on v2:172.21.15.33:6834/3594813995 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1a798c0 0x1a7bca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c23561457437c4a4 server_cookie=f83e8d91c48a72fa in_seq=0 out_seq=0 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 1 -- 172.21.15.33:0/3600866294 <== mds.0 v2:172.21.15.33:6834/3594813995 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x19393e0 con 0x1a798c0 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 10 client.5340 handle_command_reply: tid=0 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 client.5340 shutdown 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 msgr2=0x1a52c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 0x1a52c00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x1939000 tx=0x7f36e8017000 comp rx=0 tx=0).stop 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 shutdown_connections 2023-10-06T02:46:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1a798c0 0x1a7bca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x183a790 0x1a53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a53140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 --2- 172.21.15.33:0/3600866294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17ef710 0x1a52c00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36e57fa700 0 client.5340 ms_handle_reset on v2:172.21.15.33:6834/3594813995 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 >> 172.21.15.33:0/3600866294 conn(0x17efeb0 msgr2=0x1a298a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 shutdown_connections 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.185+0000 7f36ff891740 1 -- 172.21.15.33:0/3600866294 wait complete. 2023-10-06T02:46:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.189+0000 7f36ff891740 20 client.5340 trim_cache size 0 max 16384 2023-10-06T02:46:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 Processor -- start 2023-10-06T02:46:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- start start 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1ac6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1ac6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1ac6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x199b490 con 0x17efae0 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x196c550 con 0x183a790 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a04bb0 con 0x17ef710 2023-10-06T02:46:33.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1ac6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1ac6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41214/0 (socket says 172.21.15.33:41214) 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 -- 172.21.15.33:0/1036886786 learned_addr learned my addr 172.21.15.33:0/1036886786 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1ac6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f77fe700 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1ac6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 -- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 msgr2=0x1ac6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1ac6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 -- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 msgr2=0x1ac6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1ac6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 -- 172.21.15.33:0/1036886786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f80c55a0 con 0x17efae0 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36f67fc700 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1ac6a40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f36e0050590 tx=0x7f36e0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86b816e5251bfcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ecff9700 1 -- 172.21.15.33:0/1036886786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0063040 con 0x17efae0 2023-10-06T02:46:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ecff9700 1 -- 172.21.15.33:0/1036886786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e004e9d0 con 0x17efae0 2023-10-06T02:46:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ecff9700 1 -- 172.21.15.33:0/1036886786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0050960 con 0x17efae0 2023-10-06T02:46:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- 172.21.15.33:0/1036886786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 msgr2=0x1ac6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1ac6a40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f36e0050590 tx=0x7f36e0014040 comp rx=0 tx=0).stop 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- 172.21.15.33:0/1036886786 shutdown_connections 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x183a790 0x1ac6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x17efae0 0x1ac6a40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 --2- 172.21.15.33:0/1036886786 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1ac6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- 172.21.15.33:0/1036886786 >> 172.21.15.33:0/1036886786 conn(0x17efeb0 msgr2=0x1a52b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- 172.21.15.33:0/1036886786 shutdown_connections 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.209+0000 7f36ff891740 1 -- 172.21.15.33:0/1036886786 wait complete. 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 -- start start 2023-10-06T02:46:33.217 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 Processor -- start 2023-10-06T02:46:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a85670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a85bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x183a790 0x1a860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x196c010 con 0x183a790 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x186c650 con 0x17efae0 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x1937b80 con 0x17ef710 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36ff891740 10 client.0 _ll_register_callbacks cb 0x18b30e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x183a790 0x1a860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a85670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a85670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:60384/0 (socket says 172.21.15.33:60384) 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f6ffd700 1 -- 172.21.15.33:0/3706696829 learned_addr learned my addr 172.21.15.33:0/3706696829 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f67fc700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a85bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f77fe700 1 -- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 msgr2=0x1a85670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a85670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f77fe700 1 -- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 msgr2=0x1a85bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a85bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f77fe700 1 -- 172.21.15.33:0/3706696829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f80a47a0 con 0x183a790 2023-10-06T02:46:33.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.213+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a85670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36f77fe700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x183a790 0x1a860f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f36f0016000 tx=0x7f36f0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3054e36968e96ddf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 1 -- 172.21.15.33:0/3706696829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0021040 con 0x183a790 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 1 -- 172.21.15.33:0/3706696829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f36f002adb0 con 0x183a790 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36e800e630 con 0x183a790 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 1 -- 172.21.15.33:0/3706696829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f001c610 con 0x183a790 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 10 client.5346 fetch_fsmap learned FSMap version 16 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x194bd80 con 0x183a790 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 1 -- 172.21.15.33:0/3706696829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e800e630 con 0x183a790 2023-10-06T02:46:33.224 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 1 -- 172.21.15.33:0/3706696829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f36f001c770 con 0x183a790 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 10 client.5346 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 10 client.5346 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 20 client.5346 populate_metadata read hostname 'smithi033' 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1ad0780 0x1ad2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 4 client.5346 mds_command: new command op to 5145 tid=0[{"prefix": "status"}] 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 --> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] -- command(tid 0: {"prefix": "status"}) v1 -- 0x196b410 con 0x1ad0780 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1ad0780 0x1ad2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 10 client.5346 ms_handle_connect on v2:172.21.15.33:6834/3594813995 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36f6ffd700 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1ad0780 0x1ad2b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb6566204ece44c server_cookie=7707724237a0d110 in_seq=0 out_seq=0 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 1 -- 172.21.15.33:0/3706696829 <== mds.0 v2:172.21.15.33:6834/3594813995 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x196b410 con 0x1ad0780 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 10 client.5346 handle_command_reply: tid=0 2023-10-06T02:46:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 client.5346 shutdown 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x183a790 msgr2=0x1a860f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x183a790 0x1a860f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f36f0016000 tx=0x7f36f0019000 comp rx=0 tx=0).stop 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 shutdown_connections 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:6834/3594813995,v1:172.21.15.33:6835/3594813995] conn(0x1ad0780 0x1ad2b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x183a790 0x1a860f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x17efae0 0x1a85bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 --2- 172.21.15.33:0/3706696829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x17ef710 0x1a85670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 >> 172.21.15.33:0/3706696829 conn(0x17efeb0 msgr2=0x1a53240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36e57fa700 0 client.5346 ms_handle_reset on v2:172.21.15.33:6834/3594813995 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 shutdown_connections 2023-10-06T02:46:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 1 -- 172.21.15.33:0/3706696829 wait complete. 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.217+0000 7f36ff891740 20 client.5346 trim_cache size 0 max 16384 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "8ff12af2-25a8-465d-8914-d29bca58b4dd", 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5145, 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2023-10-06T02:46:33.227 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 4.6652541200000002, 2023-10-06T02:46:33.228 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 16, 2023-10-06T02:46:33.228 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 44, 2023-10-06T02:46:33.228 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 44, 2023-10-06T02:46:33.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.633799803 2023-10-06T02:46:33.228 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f36dc060550 msgr2=0x7f36dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f36dc060550 0x7f36dc062a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36f80b0da0 tx=0x7f36e8009040 comp rx=0 tx=0).stop 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 msgr2=0x7f36f81696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 0x7f36f81696f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c790 tx=0x7f36e0007e10 comp rx=0 tx=0).stop 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 shutdown_connections 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36f815ace0 0x7f36f8166890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36f80adb60 0x7f36f81696f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.229 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f36dc060550 0x7f36dc062a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 --2- 172.21.15.33:0/3797747302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36f8067970 0x7f36f8167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 >> 172.21.15.33:0/3797747302 conn(0x7f36f80c8240 msgr2=0x7f36f8156350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 shutdown_connections 2023-10-06T02:46:33.230 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.221+0000 7f36f7fff700 1 -- 172.21.15.33:0/3797747302 wait complete. 2023-10-06T02:46:33.239 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:46:33.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 Processor -- start 2023-10-06T02:46:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 -- start start 2023-10-06T02:46:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a01403a0 0x7fa8a0140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00afc70 0x7fa8a00c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a015b920 con 0x7fa8a015b4a0 2023-10-06T02:46:33.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00c9f70 con 0x7fa8a01403a0 2023-10-06T02:46:33.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a6c41700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00ca180 con 0x7fa8a00afc70 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a01403a0 0x7fa8a0140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a51de700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00afc70 0x7fa8a00c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa89ffff700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa8a015b4a0 0x7fa8a00af560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:57284/0 (socket says 172.21.15.33:57284) 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa8a01403a0 0x7fa8a0140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:58546/0 (socket says 172.21.15.33:58546) 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a51de700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00afc70 0x7fa8a00c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:60400/0 (socket says 172.21.15.33:60400) 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa89ffff700 1 -- 172.21.15.33:0/3469563572 learned_addr learned my addr 172.21.15.33:0/3469563572 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 -- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00afc70 msgr2=0x7fa8a00c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 --2- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00afc70 0x7fa8a00c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 -- 172.21.15.33:0/3469563572 >> v1:172.21.15.167:6789/0 conn(0x7fa8a015b4a0 legacy=0x7fa8a00af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 -- 172.21.15.33:0/3469563572 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00ca460 con 0x7fa8a01403a0 2023-10-06T02:46:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.349+0000 7fa8a49dd700 1 --2- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a0140770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098e0 tx=0x7fa8940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd950eec93f17c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3469563572 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894011630 con 0x7fa8a01403a0 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3469563572 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894011790 con 0x7fa8a01403a0 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa89f7fe700 1 -- 172.21.15.33:0/3469563572 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894011b30 con 0x7fa8a01403a0 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 msgr2=0x7fa8a0140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a0140770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098e0 tx=0x7fa8940091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- 172.21.15.33:0/3469563572 shutdown_connections 2023-10-06T02:46:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a00afc70 0x7fa8a00c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/3469563572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a0140770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- 172.21.15.33:0/3469563572 >> 172.21.15.33:0/3469563572 conn(0x7fa8a00cdb90 msgr2=0x7fa8a00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- 172.21.15.33:0/3469563572 shutdown_connections 2023-10-06T02:46:33.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- 172.21.15.33:0/3469563572 wait complete. 2023-10-06T02:46:33.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 Processor -- start 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- start start 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 0x7fa8a0154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a015b4a0 0x7fa8a0155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a013e080 con 0x7fa8a00afc70 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0166f10 con 0x7fa8a01403a0 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a6c41700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01546b0 con 0x7fa8a015b4a0 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 0x7fa8a0154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 0x7fa8a0154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41240/0 (socket says 172.21.15.33:41240) 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 -- 172.21.15.33:0/827966239 learned_addr learned my addr 172.21.15.33:0/827966239 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a51de700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a015b4a0 0x7fa8a0155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa89ffff700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 -- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a015b4a0 msgr2=0x7fa8a0155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a015b4a0 0x7fa8a0155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 -- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 msgr2=0x7fa8a013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.353+0000 7fa8a49dd700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa8a49dd700 1 -- 172.21.15.33:0/827966239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894009420 con 0x7fa8a00afc70 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa89ffff700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa8a49dd700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 0x7fa8a0154170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa8940030c0 tx=0x7fa894016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89401cdc0 con 0x7fa8a00afc70 2023-10-06T02:46:33.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89401a450 con 0x7fa8a00afc70 2023-10-06T02:46:33.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a016a6a0 con 0x7fa8a00afc70 2023-10-06T02:46:33.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89401a810 con 0x7fa8a00afc70 2023-10-06T02:46:33.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a016ac60 con 0x7fa8a00afc70 2023-10-06T02:46:33.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00710b0 con 0x7fa8a00afc70 2023-10-06T02:46:33.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.357+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa89401ac00 con 0x7fa8a00afc70 2023-10-06T02:46:33.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.361+0000 7fa89cff9700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:33.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.361+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fa89407e470 con 0x7fa8a00afc70 2023-10-06T02:46:33.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.361+0000 7fa89ffff700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:33.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.361+0000 7fa89ffff700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa890007fd0 tx=0x7fa890009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:33.372 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.369+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa89404e7c0 con 0x7fa8a00afc70 2023-10-06T02:46:33.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.565+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8a00ca740 con 0x7fa8a00afc70 2023-10-06T02:46:33.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa89cff9700 1 -- 172.21.15.33:0/827966239 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa894051de0 con 0x7fa8a00afc70 2023-10-06T02:46:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 msgr2=0x7fa88c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa890007fd0 tx=0x7fa890009040 comp rx=0 tx=0).stop 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 msgr2=0x7fa8a0154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 0x7fa8a0154170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa8940030c0 tx=0x7fa894016000 comp rx=0 tx=0).stop 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 shutdown_connections 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa88c060550 0x7fa88c062a00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a015b4a0 0x7fa8a0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a01403a0 0x7fa8a013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 --2- 172.21.15.33:0/827966239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a00afc70 0x7fa8a0154170 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 >> 172.21.15.33:0/827966239 conn(0x7fa8a00cdb90 msgr2=0x7fa8a013bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 shutdown_connections 2023-10-06T02:46:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:33.569+0000 7fa8a6c41700 1 -- 172.21.15.33:0/827966239 wait complete. 2023-10-06T02:46:33.586 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:46:33.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:46:33.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:33.587 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T02:46:33.595 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:46:33.595 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:46:33.595 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T02:46:33.596 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42518sec preferred_lft 42518sec 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:72:46:44:8b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:46:33.597 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 92:7d:32:8e:0c:d8 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::5ca2:30ff:feae:fa92/64 scope link 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ee:c8:8e:78:96:20 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ecc8:8eff:fe78:9620/64 scope link 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 92:7d:32:8e:0c:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:46:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::907d:32ff:fe8e:cd8/64 scope link 2023-10-06T02:46:33.599 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:33.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:33.599 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:46:33.610 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T02:46:33.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:33.612 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T02:46:33.668 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T02:46:33.668 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:46:33.668 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T02:46:33.668 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:46:33.669 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T02:46:33.669 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:46:33.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:33.669 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:46:33.670 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:46:33.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:46:33.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:46:33.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:46:33.670 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:46:33.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:33.710 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:46:33.716 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:46:33.717 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:46:33.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:46:33.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:33.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T02:46:33.731 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T02:46:33.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:46:33.732 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:33.738 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:46:33.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:46:33.738 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:33.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:46:33.806 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:33.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:33.821 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:33.822 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:33.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:33.823 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:46:33.830 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T02:46:33.830 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T02:46:33.833 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:46:33.847 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:33.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:33.907 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:33.908 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:33.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:33.909 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:46:33.918 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2023-10-06T02:46:33.913+0000 7f1089587380 -1 init, newargv = 0x55a992566c30 newargc=15 2023-10-06T02:46:33.918 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[43864]: starting ceph client 2023-10-06T02:46:33.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[43864]: starting fuse 2023-10-06T02:46:35.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:46:35.034 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:35.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:35.047 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:35.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:35.047 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:46:35.098 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:46:35.098 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T02:46:35.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> import re 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2023-10-06T02:46:35.099 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> return f 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:35.100 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2023-10-06T02:46:35.101 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:46:35.183 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.43864.asok 2023-10-06T02:46:35.367 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43864.asok 2023-10-06T02:46:35.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:35.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43864.asok status 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T02:46:35.468 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "43864", 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5355, 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5355 2023-10-06T02:46:35.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3406748255 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3406748255 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.5355 192.168.0.1:0/3406748255", 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/3406748255", 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 16, 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 44, 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2023-10-06T02:46:35.471 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:46:35.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:35.478 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2023-10-06T02:46:35.478 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:35.483 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2023-10-06T02:46:35.483 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:35.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:35.484 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:35.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:35.501 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir mydir) 2023-10-06T02:46:35.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:35.520 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch mydir/alpha) 2023-10-06T02:46:35.577 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:46:35.577 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:46:35.577 DEBUG:teuthology.orchestra.run.smithi033:> import stat 2023-10-06T02:46:35.577 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:35.577 DEBUG:teuthology.orchestra.run.smithi033:> print(os.stat("/home/ubuntu/cephtest/mnt.0/mydir").st_ino) 2023-10-06T02:46:35.577 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:46:35.645 INFO:teuthology.orchestra.run.smithi033.stdout:1099511627776 2023-10-06T02:46:35.834 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:46:35.834 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:46:35.834 DEBUG:teuthology.orchestra.run.smithi033:> import stat 2023-10-06T02:46:35.834 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:35.834 DEBUG:teuthology.orchestra.run.smithi033:> print(os.stat("/home/ubuntu/cephtest/mnt.0/mydir/alpha").st_ino) 2023-10-06T02:46:35.834 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:46:35.905 INFO:teuthology.orchestra.run.smithi033.stdout:1099511627777 2023-10-06T02:46:36.093 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T02:46:36.093 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:46:36.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:36.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[43864]: fuse finished with error 0 and tester_r 0 2023-10-06T02:46:36.110 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T02:46:42.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:46:42.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:42.213 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:46:42.221 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2023-10-06T02:46:42.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 Processor -- start 2023-10-06T02:46:42.301 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 -- start start 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd55b0d0 0x5632dd554fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 0x5632dd5ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e3970 0x5632dd557220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dd45c840 con 0x5632dd5e35a0 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dd45c4e0 con 0x5632dd55b0d0 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd627d3ce80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632dd453270 con 0x5632dd5e3970 2023-10-06T02:46:42.304 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd625072700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 0x5632dd5ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd625072700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 0x5632dd5ec120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41256/0 (socket says 172.21.15.33:41256) 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.297+0000 7fd625072700 1 -- 172.21.15.33:0/1315673903 learned_addr learned my addr 172.21.15.33:0/1315673903 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625873700 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd55b0d0 0x5632dd554fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd626074700 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e3970 0x5632dd557220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625072700 1 -- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e3970 msgr2=0x5632dd557220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625072700 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e3970 0x5632dd557220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625072700 1 -- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd55b0d0 msgr2=0x5632dd554fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625072700 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd55b0d0 0x5632dd554fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625072700 1 -- 172.21.15.33:0/1315673903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62000a0e0 con 0x5632dd5e35a0 2023-10-06T02:46:42.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625873700 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd55b0d0 0x5632dd554fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:42.310 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd625072700 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 0x5632dd5ec120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd610001c90 tx=0x7fd610001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca066b59aee9ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd61ffff700 1 -- 172.21.15.33:0/1315673903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000bbd0 con 0x5632dd5e35a0 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd61ffff700 1 -- 172.21.15.33:0/1315673903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61000bd50 con 0x5632dd5e35a0 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd61ffff700 1 -- 172.21.15.33:0/1315673903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000e6c0 con 0x5632dd5e35a0 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 -- 172.21.15.33:0/1315673903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 msgr2=0x5632dd5ec120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 0x5632dd5ec120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd610001c90 tx=0x7fd610001770 comp rx=0 tx=0).stop 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 -- 172.21.15.33:0/1315673903 shutdown_connections 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e3970 0x5632dd557220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd5e35a0 0x5632dd5ec120 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/1315673903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd55b0d0 0x5632dd554fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 -- 172.21.15.33:0/1315673903 >> 172.21.15.33:0/1315673903 conn(0x5632dd5172f0 msgr2=0x5632dd5eadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 -- 172.21.15.33:0/1315673903 shutdown_connections 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 -- 172.21.15.33:0/1315673903 wait complete. 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 Processor -- start 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.301+0000 7fd627d3ce80 1 -- start start 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd55b0d0 0x5632dd5d9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e35a0 0x5632dd5f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd5e3970 0x5632dd5f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dd5f5530 con 0x5632dd55b0d0 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dd5f4d40 con 0x5632dd5e3970 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632dd5dd5e0 con 0x5632dd5e35a0 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd55b0d0 0x5632dd5d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd55b0d0 0x5632dd5d9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41266/0 (socket says 172.21.15.33:41266) 2023-10-06T02:46:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 -- 172.21.15.33:0/194722311 learned_addr learned my addr 172.21.15.33:0/194722311 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd626074700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd5e3970 0x5632dd5f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625072700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e35a0 0x5632dd5f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 -- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e35a0 msgr2=0x5632dd5f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5632dd5e35a0 0x5632dd5f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 -- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd5e3970 msgr2=0x5632dd5f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5632dd5e3970 0x5632dd5f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 -- 172.21.15.33:0/194722311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632dd485040 con 0x5632dd55b0d0 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd625873700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd55b0d0 0x5632dd5d9500 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd620005ac0 tx=0x7fd620014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd61e7fc700 1 -- 172.21.15.33:0/194722311 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620003310 con 0x5632dd55b0d0 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd61e7fc700 1 -- 172.21.15.33:0/194722311 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd620011690 con 0x5632dd55b0d0 2023-10-06T02:46:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd61e7fc700 1 -- 172.21.15.33:0/194722311 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620011a50 con 0x5632dd55b0d0 2023-10-06T02:46:42.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd614001e10 con 0x5632dd55b0d0 2023-10-06T02:46:42.314 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632dd46b250 con 0x5632dd55b0d0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd61e7fc700 1 -- 172.21.15.33:0/194722311 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd620024070 con 0x5632dd55b0d0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.305+0000 7fd61e7fc700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd60805eeb0 0x7fd608061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd61e7fc700 1 -- 172.21.15.33:0/194722311 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd62007d800 con 0x5632dd55b0d0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5632dd545060 0x5632dd545470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5632dd5f30f0 con 0x5632dd545060 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd625072700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd60805eeb0 0x7fd608061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd626074700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5632dd545060 0x5632dd545470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd626074700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5632dd545060 0x5632dd545470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd625072700 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd60805eeb0 0x7fd608061360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6100017b0 tx=0x7fd610014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.316 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd626074700 1 -- 172.21.15.33:0/194722311 <== osd.0 v2:172.21.15.33:6816/33421 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd618009be0 con 0x5632dd545060 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:error getting omap key set cephfs_metadata/10000000000.00000000: (2) No such file or directory 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5632dd545060 msgr2=0x5632dd545470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5632dd545060 0x5632dd545470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd60805eeb0 msgr2=0x7fd608061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd60805eeb0 0x7fd608061360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6100017b0 tx=0x7fd610014000 comp rx=0 tx=0).stop 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd55b0d0 msgr2=0x5632dd5d9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 --2- 172.21.15.33:0/194722311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632dd55b0d0 0x5632dd5d9500 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd620005ac0 tx=0x7fd620014000 comp rx=0 tx=0).stop 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd626074700 1 -- 172.21.15.33:0/194722311 reap_dead start 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 shutdown_connections 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 >> 172.21.15.33:0/194722311 conn(0x5632dd5172f0 msgr2=0x5632dd5571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 shutdown_connections 2023-10-06T02:46:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.309+0000 7fd627d3ce80 1 -- 172.21.15.33:0/194722311 wait complete. 2023-10-06T02:46:42.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:42.318 ERROR:tasks.cephfs.filesystem:Command failed on smithi033 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000' 2023-10-06T02:46:42.319 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-10-06T02:46:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 Processor -- start 2023-10-06T02:46:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 -- start start 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51ddd4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e5a0 0x55c51de5cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e970 0x55c51de5d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c51dd33220 con 0x55c51ddd60d0 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c51dcd7840 con 0x55c51de5e5a0 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c51dcd74e0 con 0x55c51de5e970 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51ddd4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51ddd4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41268/0 (socket says 172.21.15.33:41268) 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 -- 172.21.15.33:0/1473446285 learned_addr learned my addr 172.21.15.33:0/1473446285 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba126be700 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e970 0x55c51de5d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba116bc700 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e5a0 0x55c51de5cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 -- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e970 msgr2=0x55c51de5d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e970 0x55c51de5d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 -- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e5a0 msgr2=0x55c51de5cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e5a0 0x55c51de5cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 -- 172.21.15.33:0/1473446285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c005cf0 con 0x55c51ddd60d0 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba116bc700 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e5a0 0x55c51de5cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba11ebd700 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51ddd4470 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba04006fc0 tx=0x7fba04006a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634d3653371fdb18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba03fff700 1 -- 172.21.15.33:0/1473446285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04006110 con 0x55c51ddd60d0 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba03fff700 1 -- 172.21.15.33:0/1473446285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04006290 con 0x55c51ddd60d0 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba03fff700 1 -- 172.21.15.33:0/1473446285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04013670 con 0x55c51ddd60d0 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 -- 172.21.15.33:0/1473446285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 msgr2=0x55c51ddd4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.349+0000 7fba14386e80 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51ddd4470 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba04006fc0 tx=0x7fba04006a40 comp rx=0 tx=0).stop 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- 172.21.15.33:0/1473446285 shutdown_connections 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e970 0x55c51de5d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e5a0 0x55c51de5cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 --2- 172.21.15.33:0/1473446285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51ddd4470 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- 172.21.15.33:0/1473446285 >> 172.21.15.33:0/1473446285 conn(0x55c51dd922f0 msgr2=0x55c51dd94430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- 172.21.15.33:0/1473446285 shutdown_connections 2023-10-06T02:46:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- 172.21.15.33:0/1473446285 wait complete. 2023-10-06T02:46:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 Processor -- start 2023-10-06T02:46:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- start start 2023-10-06T02:46:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51de67040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e5a0 0x55c51ddd3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e970 0x55c51ddbfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c51dcce270 con 0x55c51ddd60d0 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c51ddbf0a0 con 0x55c51de5e970 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c51ddbe490 con 0x55c51de5e5a0 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51de67040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51de67040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41272/0 (socket says 172.21.15.33:41272) 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 -- 172.21.15.33:0/3442075302 learned_addr learned my addr 172.21.15.33:0/3442075302 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba116bc700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e5a0 0x55c51ddd3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba126be700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e970 0x55c51ddbfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 -- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e5a0 msgr2=0x55c51ddd3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e5a0 0x55c51ddd3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 -- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e970 msgr2=0x55c51ddbfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c51de5e970 0x55c51ddbfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 -- 172.21.15.33:0/3442075302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc001e10 con 0x55c51ddd60d0 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba116bc700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c51de5e5a0 0x55c51ddd3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba11ebd700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51de67040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fba04013780 tx=0x7fba04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba027fc700 1 -- 172.21.15.33:0/3442075302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04006400 con 0x55c51ddd60d0 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba027fc700 1 -- 172.21.15.33:0/3442075302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba040036b0 con 0x55c51ddd60d0 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c51dcd9770 con 0x55c51ddd60d0 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba027fc700 1 -- 172.21.15.33:0/3442075302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04003a70 con 0x55c51ddd60d0 2023-10-06T02:46:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.353+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c51dcd91d0 con 0x55c51ddd60d0 2023-10-06T02:46:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.357+0000 7fba027fc700 1 -- 172.21.15.33:0/3442075302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fba04028460 con 0x55c51ddd60d0 2023-10-06T02:46:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.357+0000 7fba027fc700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb9f405eeb0 0x7fb9f4061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.357+0000 7fba027fc700 1 -- 172.21.15.33:0/3442075302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fba0407d190 con 0x55c51ddd60d0 2023-10-06T02:46:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba116bc700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb9f405eeb0 0x7fb9f4061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c51de67580 0x55c51de6e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55c51ddbe040 con 0x55c51de67580 2023-10-06T02:46:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba126be700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c51de67580 0x55c51de6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba116bc700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb9f405eeb0 0x7fb9f4061360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55c51de66830 tx=0x7fba08000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba126be700 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c51de67580 0x55c51de6e200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba126be700 1 -- 172.21.15.33:0/3442075302 <== osd.6 v2:172.21.15.167:6816/33656 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fba0c0078e0 con 0x55c51de67580 2023-10-06T02:46:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:error getting xattr cephfs_data/10000000001.00000000/parent: (2) No such file or directory 2023-10-06T02:46:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c51de67580 msgr2=0x55c51de6e200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c51de67580 0x55c51de6e200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb9f405eeb0 msgr2=0x7fb9f4061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb9f405eeb0 0x7fb9f4061360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55c51de66830 tx=0x7fba08000bc0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 msgr2=0x55c51de67040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 --2- 172.21.15.33:0/3442075302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c51ddd60d0 0x55c51de67040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fba04013780 tx=0x7fba04014000 comp rx=0 tx=0).stop 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba126be700 1 -- 172.21.15.33:0/3442075302 reap_dead start 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 shutdown_connections 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 >> 172.21.15.33:0/3442075302 conn(0x55c51dd922f0 msgr2=0x55c51dd94390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 shutdown_connections 2023-10-06T02:46:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.361+0000 7fba14386e80 1 -- 172.21.15.33:0/3442075302 wait complete. 2023-10-06T02:46:42.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:42.369 ERROR:tasks.cephfs.filesystem:Command failed on smithi033 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent' 2023-10-06T02:46:42.369 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2023-10-06T02:46:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.397+0000 7ff1d594de80 1 Processor -- start 2023-10-06T02:46:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- start start 2023-10-06T02:46:42.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f336cc0 0x55ce0f3381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 0x55ce0f409540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0f284840 con 0x55ce0f411d10 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0f2844e0 con 0x55ce0f40b4f0 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce0f27b270 con 0x55ce0f336cc0 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3c85700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 0x55ce0f409540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f336cc0 0x55ce0f3381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3c85700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 0x55ce0f409540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41286/0 (socket says 172.21.15.33:41286) 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3c85700 1 -- 172.21.15.33:0/3922300158 learned_addr learned my addr 172.21.15.33:0/3922300158 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d2c83700 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 -- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f40b4f0 msgr2=0x55ce0f409000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 -- 172.21.15.33:0/3922300158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 msgr2=0x55ce0f409540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 0x55ce0f409540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 -- 172.21.15.33:0/3922300158 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc00a0e0 con 0x55ce0f336cc0 2023-10-06T02:46:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3c85700 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 0x55ce0f409540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d3484700 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f336cc0 0x55ce0f3381c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55ce0f33e290 tx=0x7ff1c4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5798fdb09cf291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d1c81700 1 -- 172.21.15.33:0/3922300158 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4000bc0 con 0x55ce0f336cc0 2023-10-06T02:46:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d1c81700 1 -- 172.21.15.33:0/3922300158 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c40089d0 con 0x55ce0f336cc0 2023-10-06T02:46:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d1c81700 1 -- 172.21.15.33:0/3922300158 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4025460 con 0x55ce0f336cc0 2023-10-06T02:46:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f336cc0 msgr2=0x55ce0f3381c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f336cc0 0x55ce0f3381c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55ce0f33e290 tx=0x7ff1c4016000 comp rx=0 tx=0).stop 2023-10-06T02:46:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- 172.21.15.33:0/3922300158 shutdown_connections 2023-10-06T02:46:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f411d10 0x55ce0f409540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3922300158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f336cc0 0x55ce0f3381c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- 172.21.15.33:0/3922300158 >> 172.21.15.33:0/3922300158 conn(0x55ce0f385930 msgr2=0x55ce0f40aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.401+0000 7ff1d594de80 1 -- 172.21.15.33:0/3922300158 shutdown_connections 2023-10-06T02:46:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 -- 172.21.15.33:0/3922300158 wait complete. 2023-10-06T02:46:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 Processor -- start 2023-10-06T02:46:42.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 -- start start 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f336cc0 0x55ce0f401850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f411d10 0x55ce0f409b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0f40ce40 con 0x55ce0f40b4f0 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0f37e700 con 0x55ce0f336cc0 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.405+0000 7ff1d594de80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce0f37daf0 con 0x55ce0f411d10 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d3484700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f336cc0 0x55ce0f401850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41298/0 (socket says 172.21.15.33:41298) 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 -- 172.21.15.33:0/3454868366 learned_addr learned my addr 172.21.15.33:0/3454868366 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d3c85700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f411d10 0x55ce0f409b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 -- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f411d10 msgr2=0x55ce0f409b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f411d10 0x55ce0f409b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 -- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f336cc0 msgr2=0x55ce0f401850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f336cc0 0x55ce0f401850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 -- 172.21.15.33:0/3454868366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc00a510 con 0x55ce0f40b4f0 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d3c85700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55ce0f411d10 0x55ce0f409b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d3484700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce0f336cc0 0x55ce0f401850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d2c83700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00efd0 tx=0x7ff1bc001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1cbfff700 1 -- 172.21.15.33:0/3454868366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00cdf0 con 0x55ce0f40b4f0 2023-10-06T02:46:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1cbfff700 1 -- 172.21.15.33:0/3454868366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc005070 con 0x55ce0f40b4f0 2023-10-06T02:46:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce0f2ad040 con 0x55ce0f40b4f0 2023-10-06T02:46:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1cbfff700 1 -- 172.21.15.33:0/3454868366 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc010650 con 0x55ce0f40b4f0 2023-10-06T02:46:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.409+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4005db0 con 0x55ce0f40b4f0 2023-10-06T02:46:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1cbfff700 1 -- 172.21.15.33:0/3454868366 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff1bc0109c0 con 0x55ce0f40b4f0 2023-10-06T02:46:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1cbfff700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff1b405eee0 0x7ff1b4061390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1cbfff700 1 -- 172.21.15.33:0/3454868366 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff1bc008070 con 0x55ce0f40b4f0 2023-10-06T02:46:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d3484700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff1b405eee0 0x7ff1b4061390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x55ce0f37eb90 0x55ce0f40e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- osd_op(unknown.0.0:1 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55ce0f40e8e0 con 0x55ce0f37eb90 2023-10-06T02:46:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d3c85700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x55ce0f37eb90 0x55ce0f40e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d3484700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff1b405eee0 0x7ff1b4061390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4024000 tx=0x7ff1c401d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d3c85700 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x55ce0f37eb90 0x55ce0f40e3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d3c85700 1 -- 172.21.15.33:0/3454868366 <== osd.3 v2:172.21.15.33:6817/33424 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+5 (crc 0 0 0) 0x7ff1cc0136c0 con 0x55ce0f37eb90 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x55ce0f37eb90 msgr2=0x55ce0f40e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x55ce0f37eb90 0x55ce0f40e3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff1b405eee0 msgr2=0x7ff1b4061390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff1b405eee0 0x7ff1b4061390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4024000 tx=0x7ff1c401d000 comp rx=0 tx=0).stop 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f40b4f0 msgr2=0x55ce0f409640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d594de80 1 --2- 172.21.15.33:0/3454868366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce0f40b4f0 0x55ce0f409640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00efd0 tx=0x7ff1bc001810 comp rx=0 tx=0).stop 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.413+0000 7ff1d3c85700 1 -- 172.21.15.33:0/3454868366 reap_dead start 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.417+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 shutdown_connections 2023-10-06T02:46:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.417+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 >> 172.21.15.33:0/3454868366 conn(0x55ce0f385930 msgr2=0x55ce0f338e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.417+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 shutdown_connections 2023-10-06T02:46:42.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.417+0000 7ff1d594de80 1 -- 172.21.15.33:0/3454868366 wait complete. 2023-10-06T02:46:42.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:42.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f528d6da700 1 Processor -- start 2023-10-06T02:46:42.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f528d6da700 1 -- start start 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f528d6da700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5288158fc0 0x7f528815d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f528d6da700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528815d830 con 0x7f5288158bf0 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f528d6da700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52880b00a0 con 0x7f52880b0530 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f528d6da700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5288067c10 con 0x7f5288158fc0 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f5286ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5288158bf0 0x7f52880b1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51932/0 (socket says 172.21.15.33:51932) 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f5286ffd700 1 -- 172.21.15.33:0/690545809 learned_addr learned my addr 172.21.15.33:0/690545809 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:42.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f52877fe700 1 --2- 172.21.15.33:0/690545809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5288158fc0 0x7f528815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801555922 0 0) 0x7f528815d830 con 0x7f5288158bf0 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5270003570 con 0x7f5288158bf0 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836473037 0 0) 0x7f52880b00a0 con 0x7f52880b0530 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f528815d830 con 0x7f52880b0530 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f52877fe700 1 -- 172.21.15.33:0/690545809 >> v1:172.21.15.167:6789/0 conn(0x7f52880b0530 legacy=0x7f528815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f52877fe700 1 -- 172.21.15.33:0/690545809 >> v1:172.21.15.33:6789/0 conn(0x7f5288158bf0 legacy=0x7f52880b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f52877fe700 1 -- 172.21.15.33:0/690545809 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880cba20 con 0x7f5288158fc0 2023-10-06T02:46:42.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.577+0000 7f52877fe700 1 --2- 172.21.15.33:0/690545809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5288158fc0 0x7f528815d180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5280009e90 tx=0x7f5280009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c07889b5527d43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528000e070 con 0x7f5288158fc0 2023-10-06T02:46:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528000bb80 con 0x7f5288158fc0 2023-10-06T02:46:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f5285ffb700 1 -- 172.21.15.33:0/690545809 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280012430 con 0x7f5288158fc0 2023-10-06T02:46:42.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 -- 172.21.15.33:0/690545809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5288158fc0 msgr2=0x7f528815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 --2- 172.21.15.33:0/690545809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5288158fc0 0x7f528815d180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5280009e90 tx=0x7f5280009780 comp rx=0 tx=0).stop 2023-10-06T02:46:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 -- 172.21.15.33:0/690545809 shutdown_connections 2023-10-06T02:46:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 --2- 172.21.15.33:0/690545809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5288158fc0 0x7f528815d180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 -- 172.21.15.33:0/690545809 >> 172.21.15.33:0/690545809 conn(0x7f52880cfd70 msgr2=0x7f52880d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 -- 172.21.15.33:0/690545809 shutdown_connections 2023-10-06T02:46:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 -- 172.21.15.33:0/690545809 wait complete. 2023-10-06T02:46:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 Processor -- start 2023-10-06T02:46:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.581+0000 7f528d6da700 1 -- start start 2023-10-06T02:46:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5288158bf0 0x7f528814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 0x7f528815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815eef0 0x7f52881577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528815f360 con 0x7f5288158fc0 2023-10-06T02:46:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52880c7550 con 0x7f5288158bf0 2023-10-06T02:46:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5288158530 con 0x7f528815eef0 2023-10-06T02:46:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 0x7f528815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52877fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815eef0 0x7f52881577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 0x7f528815e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33120/0 (socket says 172.21.15.33:33120) 2023-10-06T02:46:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 -- 172.21.15.33:0/4243369505 learned_addr learned my addr 172.21.15.33:0/4243369505 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f5286ffd700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5288158bf0 0x7f528814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 -- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815eef0 msgr2=0x7f52881577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815eef0 0x7f52881577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 -- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5288158bf0 msgr2=0x7f528814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5288158bf0 0x7f528814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 -- 172.21.15.33:0/4243369505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5280009a20 con 0x7f5288158fc0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f52867fc700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 0x7f528815e9b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f527800c580 tx=0x7f5278007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278010510 con 0x7f5288158fc0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5278010670 con 0x7f5288158fc0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52880c6f00 con 0x7f5288158fc0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278010a30 con 0x7f5288158fc0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.585+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528814f5f0 con 0x7f5288158fc0 2023-10-06T02:46:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.589+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52680052b0 con 0x7f5288158fc0 2023-10-06T02:46:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.589+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f527801b400 con 0x7f5288158fc0 2023-10-06T02:46:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.589+0000 7f526effd700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5274060550 0x7f5274062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:42.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.589+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f527807c570 con 0x7f5288158fc0 2023-10-06T02:46:42.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.589+0000 7f5286ffd700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5274060550 0x7f5274062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:42.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.589+0000 7f5286ffd700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5274060550 0x7f5274062a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f527c007fd0 tx=0x7f527c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:42.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.597+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f527804c8c0 con 0x7f5288158fc0 2023-10-06T02:46:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.797+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5268005560 con 0x7f5288158fc0 2023-10-06T02:46:42.802 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.797+0000 7f526effd700 1 -- 172.21.15.33:0/4243369505 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f527804fee0 con 0x7f5288158fc0 2023-10-06T02:46:42.802 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:42.802 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5274060550 msgr2=0x7f5274062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5274060550 0x7f5274062a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f527c007fd0 tx=0x7f527c009040 comp rx=0 tx=0).stop 2023-10-06T02:46:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 msgr2=0x7f528815e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 0x7f528815e9b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f527800c580 tx=0x7f5278007e10 comp rx=0 tx=0).stop 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 shutdown_connections 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5274060550 0x7f5274062a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815eef0 0x7f52881577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5288158fc0 0x7f528815e9b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 --2- 172.21.15.33:0/4243369505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5288158bf0 0x7f528814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 >> 172.21.15.33:0/4243369505 conn(0x7f52880cfd70 msgr2=0x7f52880cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 shutdown_connections 2023-10-06T02:46:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:42.801+0000 7f528d6da700 1 -- 172.21.15.33:0/4243369505 wait complete. 2023-10-06T02:46:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:42.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-10-06T02:46:42.943 INFO:teuthology.orchestra.run.smithi033.stdout:{"message":"","return_code":0} 2023-10-06T02:46:43.307 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T02:46:43.311 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2023-10-06T02:46:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 Processor -- start 2023-10-06T02:46:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 -- start start 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x56508013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x5650800c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650800c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56507ffbf840 con 0x565080146970 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56507ffbf4e0 con 0x5650800be0d0 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bdfb9e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56507ffb6270 con 0x5650801465a0 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650800c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650800c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33136/0 (socket says 172.21.15.33:33136) 2023-10-06T02:46:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 -- 172.21.15.33:0/3084808132 learned_addr learned my addr 172.21.15.33:0/3084808132 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bb2ef700 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x5650800c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bbaf0700 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x56508013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 -- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 msgr2=0x5650800c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x5650800c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 -- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 msgr2=0x56508013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x56508013ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.345+0000 7f05bc2f1700 1 -- 172.21.15.33:0/3084808132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b400a0e0 con 0x565080146970 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bc2f1700 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650800c24e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05b400abc0 tx=0x7f05b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad85c4039bfb32b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05ba2ed700 1 -- 172.21.15.33:0/3084808132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4003300 con 0x565080146970 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05ba2ed700 1 -- 172.21.15.33:0/3084808132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400f860 con 0x565080146970 2023-10-06T02:46:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05ba2ed700 1 -- 172.21.15.33:0/3084808132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b40246c0 con 0x565080146970 2023-10-06T02:46:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/3084808132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 msgr2=0x5650800c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650800c24e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05b400abc0 tx=0x7f05b4015000 comp rx=0 tx=0).stop 2023-10-06T02:46:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/3084808132 shutdown_connections 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x5650800c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650800c24e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/3084808132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x56508013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/3084808132 >> 172.21.15.33:0/3084808132 conn(0x56508007a2f0 msgr2=0x5650800bc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/3084808132 shutdown_connections 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/3084808132 wait complete. 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 Processor -- start 2023-10-06T02:46:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.349+0000 7f05bdfb9e80 1 -- start start 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x5650800bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x565080142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650801362f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565080144320 con 0x565080146970 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x565080143710 con 0x5650800be0d0 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650801439f0 con 0x5650801465a0 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650801362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650801362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33140/0 (socket says 172.21.15.33:33140) 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 -- 172.21.15.33:0/365279127 learned_addr learned my addr 172.21.15.33:0/365279127 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bb2ef700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x565080142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bbaf0700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x5650800bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 -- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 msgr2=0x565080142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x565080142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 -- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 msgr2=0x5650800bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650800be0d0 0x5650800bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 -- 172.21.15.33:0/365279127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56507ffe8040 con 0x565080146970 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bb2ef700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5650801465a0 0x565080142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bc2f1700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650801362f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05b400f6c0 tx=0x7f05b401d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05b8aea700 1 -- 172.21.15.33:0/365279127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4029070 con 0x565080146970 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05b8aea700 1 -- 172.21.15.33:0/365279127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400f9d0 con 0x565080146970 2023-10-06T02:46:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56507ffce250 con 0x565080146970 2023-10-06T02:46:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05b8aea700 1 -- 172.21.15.33:0/365279127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4024e90 con 0x565080146970 2023-10-06T02:46:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.353+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56507ffc1770 con 0x565080146970 2023-10-06T02:46:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05b8aea700 1 -- 172.21.15.33:0/365279127 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f05b4021070 con 0x565080146970 2023-10-06T02:46:43.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05b8aea700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f059c05eee0 0x7f059c061390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.362 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05b8aea700 1 -- 172.21.15.33:0/365279127 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f05b407e440 con 0x565080146970 2023-10-06T02:46:43.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05bbaf0700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f059c05eee0 0x7f059c061390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5650801447b0 0x565080144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.363 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650801450f0 con 0x5650801447b0 2023-10-06T02:46:43.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.357+0000 7f05bb2ef700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5650801447b0 0x565080144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bbaf0700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f059c05eee0 0x7f059c061390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac001220 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bb2ef700 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5650801447b0 0x565080144bb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bb2ef700 1 -- 172.21.15.33:0/365279127 <== osd.0 v2:172.21.15.33:6816/33421 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=19b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+19 (crc 0 0 0) 0x7f05a4012040 con 0x5650801447b0 2023-10-06T02:46:43.365 INFO:teuthology.orchestra.run.smithi033.stdout:alpha_head 2023-10-06T02:46:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5650801447b0 msgr2=0x565080144bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5650801447b0 0x565080144bb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f059c05eee0 msgr2=0x7f059c061390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f059c05eee0 0x7f059c061390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac001220 comp rx=0 tx=0).stop 2023-10-06T02:46:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 msgr2=0x5650801362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 --2- 172.21.15.33:0/365279127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565080146970 0x5650801362f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05b400f6c0 tx=0x7f05b401d070 comp rx=0 tx=0).stop 2023-10-06T02:46:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bc2f1700 1 -- 172.21.15.33:0/365279127 reap_dead start 2023-10-06T02:46:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 shutdown_connections 2023-10-06T02:46:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 >> 172.21.15.33:0/365279127 conn(0x56508007a2f0 msgr2=0x5650800c2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 shutdown_connections 2023-10-06T02:46:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.361+0000 7f05bdfb9e80 1 -- 172.21.15.33:0/365279127 wait complete. 2023-10-06T02:46:43.368 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2023-10-06T02:46:43.402 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.397+0000 7effeee06e80 1 Processor -- start 2023-10-06T02:46:43.403 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.397+0000 7effeee06e80 1 -- start start 2023-10-06T02:46:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeee06e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5fee1f60 0x557b5ff6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeee06e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6ad20 0x557b5ff6d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeee06e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 0x557b5ff6d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeee06e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5fde0840 con 0x557b5ff6b0f0 2023-10-06T02:46:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeee06e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5fde04e0 con 0x557b5fee1f60 2023-10-06T02:46:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeee06e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b5fdd7270 con 0x557b5ff6ad20 2023-10-06T02:46:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 0x557b5ff6d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effec13c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6ad20 0x557b5ff6d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 0x557b5ff6d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33152/0 (socket says 172.21.15.33:33152) 2023-10-06T02:46:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 -- 172.21.15.33:0/2821319519 learned_addr learned my addr 172.21.15.33:0/2821319519 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effec93d700 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5fee1f60 0x557b5ff6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effec13c700 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6ad20 0x557b5ff6d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55618/0 (socket says 172.21.15.33:55618) 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 -- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6ad20 msgr2=0x557b5ff6d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6ad20 0x557b5ff6d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 -- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5fee1f60 msgr2=0x557b5ff6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5fee1f60 0x557b5ff6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 -- 172.21.15.33:0/2821319519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd400a0e0 con 0x557b5ff6b0f0 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effed13e700 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 0x557b5ff6d7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7effdc009760 tx=0x7effdc00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c355ed9bef709a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeb13a700 1 -- 172.21.15.33:0/2821319519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc00f040 con 0x557b5ff6b0f0 2023-10-06T02:46:43.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeb13a700 1 -- 172.21.15.33:0/2821319519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc008700 con 0x557b5ff6b0f0 2023-10-06T02:46:43.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.401+0000 7effeb13a700 1 -- 172.21.15.33:0/2821319519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc008aa0 con 0x557b5ff6b0f0 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 -- 172.21.15.33:0/2821319519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 msgr2=0x557b5ff6d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 0x557b5ff6d7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7effdc009760 tx=0x7effdc00c040 comp rx=0 tx=0).stop 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 -- 172.21.15.33:0/2821319519 shutdown_connections 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6ad20 0x557b5ff6d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5ff6b0f0 0x557b5ff6d7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 --2- 172.21.15.33:0/2821319519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5fee1f60 0x557b5ff6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 -- 172.21.15.33:0/2821319519 >> 172.21.15.33:0/2821319519 conn(0x557b5fe97c90 msgr2=0x557b5fec42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 -- 172.21.15.33:0/2821319519 shutdown_connections 2023-10-06T02:46:43.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 -- 172.21.15.33:0/2821319519 wait complete. 2023-10-06T02:46:43.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 Processor -- start 2023-10-06T02:46:43.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.405+0000 7effeee06e80 1 -- start start 2023-10-06T02:46:43.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5fee1f60 0x557b5ff623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5ff6ad20 0x557b5ff7ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6b0f0 0x557b5ff7f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ff618b0 con 0x557b5fee1f60 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ff61420 con 0x557b5ff6ad20 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b5ff60810 con 0x557b5ff6b0f0 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5fee1f60 0x557b5ff623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5fee1f60 0x557b5ff623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33160/0 (socket says 172.21.15.33:33160) 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 -- 172.21.15.33:0/2717188328 learned_addr learned my addr 172.21.15.33:0/2717188328 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec13c700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5ff6ad20 0x557b5ff7ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effed13e700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6b0f0 0x557b5ff7f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 -- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6b0f0 msgr2=0x557b5ff7f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6b0f0 0x557b5ff7f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 -- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5ff6ad20 msgr2=0x557b5ff7ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5ff6ad20 0x557b5ff7ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 -- 172.21.15.33:0/2717188328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009430 con 0x557b5fee1f60 2023-10-06T02:46:43.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effed13e700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557b5ff6b0f0 0x557b5ff7f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effec93d700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5fee1f60 0x557b5ff623d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effe400c5d0 tx=0x7effe40017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effe9937700 1 -- 172.21.15.33:0/2717188328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4010400 con 0x557b5fee1f60 2023-10-06T02:46:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effe9937700 1 -- 172.21.15.33:0/2717188328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4010560 con 0x557b5fee1f60 2023-10-06T02:46:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effe9937700 1 -- 172.21.15.33:0/2717188328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4010900 con 0x557b5fee1f60 2023-10-06T02:46:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557b5fdef250 con 0x557b5fee1f60 2023-10-06T02:46:43.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b5fde2770 con 0x557b5fee1f60 2023-10-06T02:46:43.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.409+0000 7effe9937700 1 -- 172.21.15.33:0/2717188328 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7effe4010c70 con 0x557b5fee1f60 2023-10-06T02:46:43.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effe9937700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7effcc05eeb0 0x7effcc061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effe9937700 1 -- 172.21.15.33:0/2717188328 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7effe4008030 con 0x557b5fee1f60 2023-10-06T02:46:43.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effec13c700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7effcc05eeb0 0x7effcc061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x557b5ff62910 0x557b5ff69770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- osd_op(unknown.0.0:1 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x557b5ff63c90 con 0x557b5ff62910 2023-10-06T02:46:43.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effed13e700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x557b5ff62910 0x557b5ff69770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effec13c700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7effcc05eeb0 0x7effcc061360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7effd4005d80 tx=0x7effd4015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effed13e700 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x557b5ff62910 0x557b5ff69770 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effed13e700 1 -- 172.21.15.33:0/2717188328 <== osd.3 v2:172.21.15.33:6817/33424 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=19b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+19 (crc 0 0 0) 0x7effdc019020 con 0x557b5ff62910 2023-10-06T02:46:43.418 INFO:teuthology.orchestra.run.smithi033.stdout:mydir_head 2023-10-06T02:46:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x557b5ff62910 msgr2=0x557b5ff69770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x557b5ff62910 0x557b5ff69770 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7effcc05eeb0 msgr2=0x7effcc061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7effcc05eeb0 0x7effcc061360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7effd4005d80 tx=0x7effd4015000 comp rx=0 tx=0).stop 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5fee1f60 msgr2=0x557b5ff623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 --2- 172.21.15.33:0/2717188328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b5fee1f60 0x557b5ff623d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effe400c5d0 tx=0x7effe40017f0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effed13e700 1 -- 172.21.15.33:0/2717188328 reap_dead start 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 shutdown_connections 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 >> 172.21.15.33:0/2717188328 conn(0x557b5fe97c90 msgr2=0x557b5ff6b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 shutdown_connections 2023-10-06T02:46:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.413+0000 7effeee06e80 1 -- 172.21.15.33:0/2717188328 wait complete. 2023-10-06T02:46:43.421 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-10-06T02:46:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.453+0000 7f6df8438e80 1 Processor -- start 2023-10-06T02:46:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.453+0000 7f6df8438e80 1 -- start start 2023-10-06T02:46:43.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df8438e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0100d0 0x56455a00b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df8438e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a0985a0 0x56455a00acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df8438e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 0x564559ff4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df8438e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564559f11840 con 0x56455a098970 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df8438e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564559f114e0 con 0x56455a0100d0 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df8438e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564559f08270 con 0x56455a0985a0 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 0x564559ff4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df576e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a0985a0 0x56455a00acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 0x564559ff4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33174/0 (socket says 172.21.15.33:33174) 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df5f6f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0100d0 0x56455a00b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 -- 172.21.15.33:0/3398256291 learned_addr learned my addr 172.21.15.33:0/3398256291 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 -- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a0985a0 msgr2=0x56455a00acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a0985a0 0x56455a00acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 -- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0100d0 msgr2=0x56455a00b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0100d0 0x56455a00b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 -- 172.21.15.33:0/3398256291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8005c90 con 0x56455a098970 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df576e700 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a0985a0 0x56455a00acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6df6770700 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 0x564559ff4d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6de000a520 tx=0x7f6de0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2873146614b4c446 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6deffff700 1 -- 172.21.15.33:0/3398256291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de001a070 con 0x56455a098970 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6deffff700 1 -- 172.21.15.33:0/3398256291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de00136f0 con 0x56455a098970 2023-10-06T02:46:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.457+0000 7f6deffff700 1 -- 172.21.15.33:0/3398256291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0013da0 con 0x56455a098970 2023-10-06T02:46:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/3398256291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 msgr2=0x564559ff4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 0x564559ff4d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6de000a520 tx=0x7f6de0013000 comp rx=0 tx=0).stop 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/3398256291 shutdown_connections 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a0985a0 0x56455a00acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a098970 0x564559ff4d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- 172.21.15.33:0/3398256291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0100d0 0x56455a00b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/3398256291 >> 172.21.15.33:0/3398256291 conn(0x564559fcc2f0 msgr2=0x564559fce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/3398256291 shutdown_connections 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/3398256291 wait complete. 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 Processor -- start 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- start start 2023-10-06T02:46:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a0100d0 0x56455a091210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0985a0 0x564559ff4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a098970 0x564559ff5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564559ffa170 con 0x56455a0100d0 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564559ffacb0 con 0x56455a0985a0 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x564559ffa380 con 0x56455a098970 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a0100d0 0x56455a091210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a0100d0 0x56455a091210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33186/0 (socket says 172.21.15.33:33186) 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 -- 172.21.15.33:0/2852446191 learned_addr learned my addr 172.21.15.33:0/2852446191 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df6770700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a098970 0x564559ff5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df576e700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0985a0 0x564559ff4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 -- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a098970 msgr2=0x564559ff5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56455a098970 0x564559ff5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 -- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0985a0 msgr2=0x564559ff4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56455a0985a0 0x564559ff4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 -- 172.21.15.33:0/2852446191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0002000 con 0x56455a0100d0 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df5f6f700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a0100d0 0x56455a091210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6de800ac40 tx=0x7f6de800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6dee7fc700 1 -- 172.21.15.33:0/2852446191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8010050 con 0x56455a0100d0 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6dee7fc700 1 -- 172.21.15.33:0/2852446191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8011040 con 0x56455a0100d0 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564559f3a040 con 0x56455a0100d0 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6dee7fc700 1 -- 172.21.15.33:0/2852446191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8009df0 con 0x56455a0100d0 2023-10-06T02:46:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.461+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de0005cf0 con 0x56455a0100d0 2023-10-06T02:46:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.465+0000 7f6dee7fc700 1 -- 172.21.15.33:0/2852446191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6de8008690 con 0x56455a0100d0 2023-10-06T02:46:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.465+0000 7f6dee7fc700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6ddc05eeb0 0x7f6ddc061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.465+0000 7f6dee7fc700 1 -- 172.21.15.33:0/2852446191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6de807c360 con 0x56455a0100d0 2023-10-06T02:46:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.465+0000 7f6df576e700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6ddc05eeb0 0x7f6ddc061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x56455a091750 0x56455a0933f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x564559ff9330 con 0x56455a091750 2023-10-06T02:46:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df6770700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x56455a091750 0x56455a0933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df576e700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6ddc05eeb0 0x7f6ddc061360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x564559f69400 tx=0x7f6df0000f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df6770700 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x56455a091750 0x56455a0933f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df6770700 1 -- 172.21.15.33:0/2852446191 <== osd.6 v2:172.21.15.167:6816/33656 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (92) out=92b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+92 (crc 0 0 0) 0x7f6de001e610 con 0x56455a091750 2023-10-06T02:46:43.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x56455a091750 msgr2=0x56455a0933f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x56455a091750 0x56455a0933f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6ddc05eeb0 msgr2=0x7f6ddc061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6ddc05eeb0 0x7f6ddc061360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x564559f69400 tx=0x7f6df0000f40 comp rx=0 tx=0).stop 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a0100d0 msgr2=0x56455a091210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 --2- 172.21.15.33:0/2852446191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56455a0100d0 0x56455a091210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6de800ac40 tx=0x7f6de800d040 comp rx=0 tx=0).stop 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df6770700 1 -- 172.21.15.33:0/2852446191 reap_dead start 2023-10-06T02:46:43.475 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 shutdown_connections 2023-10-06T02:46:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 >> 172.21.15.33:0/2852446191 conn(0x564559fcc2f0 msgr2=0x56455a00b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 shutdown_connections 2023-10-06T02:46:43.476 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.469+0000 7f6df8438e80 1 -- 172.21.15.33:0/2852446191 wait complete. 2023-10-06T02:46:43.477 INFO:teuthology.orchestra.run.smithi033.stdout:Valphamydir 2023-10-06T02:46:43.477 DEBUG:teuthology.orchestra.run.smithi033:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 1099511627777, 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ancestors": [ 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: "dirino": 1099511627776, 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: "dname": "alpha", 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 4 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2023-10-06T02:46:43.543 INFO:teuthology.orchestra.run.smithi033.stdout: "dirino": 1, 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: "dname": "mydir", 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 6 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: "pool": 7, 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: "old_pools": [] 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:46:43.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:43.546 DEBUG:teuthology.orchestra.run.smithi033:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary 2023-10-06T02:46:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 -- start start 2023-10-06T02:46:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 Processor -- start 2023-10-06T02:46:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188300400 0x565188356580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188300800 0x565188356b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188300c00 0x565188357080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651875dfcc0 con 0x565188300800 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651875df540 con 0x565188300c00 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651875dfe00 con 0x565188300400 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38ef5a3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188300400 0x565188356580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38eeda2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188300800 0x565188356b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38efda4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188300c00 0x565188357080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38ef5a3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188300400 0x565188356580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55680/0 (socket says 172.21.15.33:55680) 2023-10-06T02:46:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38ef5a3700 1 -- 172.21.15.33:0/3045475740 learned_addr learned my addr 172.21.15.33:0/3045475740 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38efda4700 1 -- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188300400 msgr2=0x565188356580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38efda4700 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188300400 0x565188356580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38efda4700 1 -- 172.21.15.33:0/3045475740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188300800 msgr2=0x565188356b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38efda4700 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188300800 0x565188356b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.581+0000 7f38efda4700 1 -- 172.21.15.33:0/3045475740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651875c2340 con 0x565188300c00 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38efda4700 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188300c00 0x565188357080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5651883640c0 tx=0x56518832a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc4af868caeeb4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38ecd9e700 1 -- 172.21.15.33:0/3045475740 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5651875ed600 con 0x565188300c00 2023-10-06T02:46:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38ecd9e700 1 -- 172.21.15.33:0/3045475740 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651875f4600 con 0x565188300c00 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38ecd9e700 1 -- 172.21.15.33:0/3045475740 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5651875ed1e0 con 0x565188300c00 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651875c2000 con 0x565188300c00 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5651875c2ea0 con 0x565188300c00 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 4 waiting for MDS map... 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38ecd9e700 1 -- 172.21.15.33:0/3045475740 <== mon.1 v2:172.21.15.167:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x565188362a80 con 0x565188300c00 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38ecd9e700 1 -- 172.21.15.33:0/3045475740 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x565188382600 con 0x565188300c00 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 4 Got MDS map 16 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 10 main: JournalTool::main 2023-10-06T02:46:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 4 main: JournalTool: connecting to RADOS... 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 Processor -- start 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 -- start start 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301800 0x565188357600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188301c00 0x565188357b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651875dfb80 con 0x56518840c000 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651875dfcc0 con 0x565188301800 2023-10-06T02:46:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651875dedc0 con 0x565188301c00 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38eeda2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38eeda2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33206/0 (socket says 172.21.15.33:33206) 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38eeda2700 1 -- 172.21.15.33:0/1535482309 learned_addr learned my addr 172.21.15.33:0/1535482309 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38ef5a3700 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188301c00 0x565188357b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.585+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301800 0x565188357600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38eeda2700 1 -- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188301c00 msgr2=0x565188357b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188301c00 0x565188357b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38eeda2700 1 -- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301800 msgr2=0x565188357600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301800 0x565188357600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38eeda2700 1 -- 172.21.15.33:0/1535482309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651875c3860 con 0x56518840c000 2023-10-06T02:46:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 0x565188358100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5651883655f0 tx=0x565188366060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae8470883803bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38ead9a700 1 -- 172.21.15.33:0/1535482309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5651875ed080 con 0x56518840c000 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38ead9a700 1 -- 172.21.15.33:0/1535482309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651875f4300 con 0x56518840c000 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38ead9a700 1 -- 172.21.15.33:0/1535482309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5651875ed600 con 0x56518840c000 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1535482309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 msgr2=0x565188358100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 0x565188358100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5651883655f0 tx=0x565188366060 comp rx=0 tx=0).stop 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1535482309 shutdown_connections 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188301c00 0x565188357b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1535482309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301800 0x565188357600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1535482309 >> 172.21.15.33:0/1535482309 conn(0x565188301000 msgr2=0x56518832d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1535482309 shutdown_connections 2023-10-06T02:46:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1535482309 wait complete. 2023-10-06T02:46:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 Processor -- start 2023-10-06T02:46:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.589+0000 7f38f0de90c0 1 -- start start 2023-10-06T02:46:43.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301c00 0x565188358c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 0x565188358680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651875de3c0 con 0x565188301800 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651875de000 con 0x565188301c00 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651875df180 con 0x56518840c000 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 0x565188358680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 0x565188358680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33208/0 (socket says 172.21.15.33:33208) 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 -- 172.21.15.33:0/1145717767 learned_addr learned my addr 172.21.15.33:0/1145717767 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301c00 0x565188358c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 -- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56518840c000 msgr2=0x565188358100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x56518840c000 0x565188358100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 -- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301c00 msgr2=0x565188358c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188301c00 0x565188358c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651875c29c0 con 0x565188301800 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38ef5a3700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 0x565188358680 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x56518838a0f0 tx=0x5651883b2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38e9597700 1 -- 172.21.15.33:0/1145717767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5651875ed080 con 0x565188301800 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38e9597700 1 -- 172.21.15.33:0/1145717767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5651875f4d80 con 0x565188301800 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5651875c3a00 con 0x565188301800 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38e9597700 1 -- 172.21.15.33:0/1145717767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5651875ed600 con 0x565188301800 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651875c29c0 con 0x565188301800 2023-10-06T02:46:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.593+0000 7f38f0de90c0 4 main: JournalTool: resolving pool 6 2023-10-06T02:46:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38e9597700 1 -- 172.21.15.33:0/1145717767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x56518830f080 con 0x565188301800 2023-10-06T02:46:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38e9597700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x56518838c400 0x565188359700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38e9597700 1 -- 172.21.15.33:0/1145717767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x565188363880 con 0x565188301800 2023-10-06T02:46:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38f0de90c0 4 main: JournalTool: creating IoCtx.. 2023-10-06T02:46:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38f0de90c0 4 main: Executing for rank 0 2023-10-06T02:46:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x56518840c400 0x565188357b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x56518838c400 0x565188359700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- osd_op(unknown.0.0:1 6.f 6:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x56518840c800 con 0x56518840c400 2023-10-06T02:46:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x56518840c400 0x565188357b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x56518838c400 0x565188359700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x565188365740 tx=0x565188366240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38eeda2700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x56518840c400 0x565188357b80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.597+0000 7f38eeda2700 1 -- 172.21.15.33:0/1145717767 <== osd.2 v2:172.21.15.33:6808/33422 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x565188360fc0 con 0x56518840c400 2023-10-06T02:46:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Pointer 400.00000000 is readable 2023-10-06T02:46:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 JournalScanner::scan: reading header object '200.00000000' 2023-10-06T02:46:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- osd_op(unknown.0.0:2 6.14 6:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x56518840cc00 con 0x56518840c400 2023-10-06T02:46:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38eeda2700 1 -- 172.21.15.33:0/1145717767 <== osd.2 v2:172.21.15.33:6808/33422 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv17 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x565188360fc0 con 0x56518840c400 2023-10-06T02:46:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 10 Header 0x400000 0x40244b 0x4027c7 2023-10-06T02:46:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 10 Starting journal scan from offset 0x40244b 2023-10-06T02:46:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- osd_op(unknown.0.0:3 6.14 6:2e2fa760:::200.00000001:head [read 9291~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x56518840d000 con 0x56518840c400 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38eeda2700 1 -- 172.21.15.33:0/1145717767 <== osd.2 v2:172.21.15.33:6808/33422 3 ==== osd_op_reply(3 200.00000001 [read 9291~892 out=892b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x565188360fc0 con 0x56518840c400 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Read 0x37c bytes from 200.00000001 gap=0 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 10 Parsing data, 0x37c bytes available 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 10 Attempting decode at 0x40244b 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 10 Consumed 0x37c bytes 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 10 Valid entry at 0x40244b 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x56518840d400 0x565188357600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- osd_op(unknown.0.0:4 6.1b 6:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x56518840d800 con 0x56518840d400 2023-10-06T02:46:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38efda4700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x56518840d400 0x565188357600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38efda4700 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x56518840d400 0x565188357600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38efda4700 1 -- 172.21.15.33:0/1145717767 <== osd.3 v2:172.21.15.33:6817/33424 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x565188361200 con 0x56518840d400 2023-10-06T02:46:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Reached end of journal objects 2023-10-06T02:46:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Scanned objects, 0 missing, 1 valid 2023-10-06T02:46:43.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Events scanned, 0 gaps 2023-10-06T02:46:43.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Found 1 valid events 2023-10-06T02:46:43.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 Selected 1 events events for processing 2023-10-06T02:46:43.608 INFO:teuthology.orchestra.run.smithi033.stdout:Events by type: 2023-10-06T02:46:43.608 INFO:teuthology.orchestra.run.smithi033.stdout: SUBTREEMAP: 1 2023-10-06T02:46:43.608 INFO:teuthology.orchestra.run.smithi033.stdout:Errors: 0 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 4 1 events 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x56518840c400 msgr2=0x565188357b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x56518840c400 0x565188357b80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x56518840d400 msgr2=0x565188357600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x56518840d400 0x565188357600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x56518838c400 msgr2=0x565188359700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x56518838c400 0x565188359700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x565188365740 tx=0x565188366240 comp rx=0 tx=0).stop 2023-10-06T02:46:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 msgr2=0x565188358680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38ef5a3700 1 -- 172.21.15.33:0/1145717767 reap_dead start 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 0x565188358680 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x56518838a0f0 tx=0x5651883b2c00 comp rx=0 tx=0).stop 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 shutdown_connections 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/1145717767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188301800 0x565188358680 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 >> 172.21.15.33:0/1145717767 conn(0x565188301000 msgr2=0x56518832d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 shutdown_connections 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.601+0000 7f38f0de90c0 1 -- 172.21.15.33:0/1145717767 wait complete. 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188300c00 msgr2=0x565188357080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188300c00 0x565188357080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5651883640c0 tx=0x56518832a4e0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 shutdown_connections 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565188300c00 0x565188357080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565188300800 0x565188356b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 --2- 172.21.15.33:0/3045475740 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x565188300400 0x565188356580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 >> 172.21.15.33:0/3045475740 conn(0x565188300000 msgr2=0x56518832d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 shutdown_connections 2023-10-06T02:46:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.605+0000 7f38f0de90c0 1 -- 172.21.15.33:0/3045475740 wait complete. 2023-10-06T02:46:43.613 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'get', 'summary'] in time 0:00:00.066467, result: Events by type: SUBTREEMAP: 1 Errors: 0 2023-10-06T02:46:43.613 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:43.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e46019700 1 Processor -- start 2023-10-06T02:46:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e46019700 1 -- start start 2023-10-06T02:46:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e46019700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e40067970 0x7f4e4015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e46019700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e40157eb0 con 0x7f4e400adf30 2023-10-06T02:46:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e46019700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e40069820 con 0x7f4e400adb60 2023-10-06T02:46:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e46019700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e40067ea0 con 0x7f4e40067970 2023-10-06T02:46:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e44816700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e40067970 0x7f4e4015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e44816700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4e40067970 0x7f4e4015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33222/0 (socket says 172.21.15.33:33222) 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e45818700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4e400adf30 0x7f4e40069cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51962/0 (socket says 172.21.15.33:51962) 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.725+0000 7f4e44816700 1 -- 172.21.15.33:0/2592067756 learned_addr learned my addr 172.21.15.33:0/2592067756 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e37fff700 1 -- 172.21.15.33:0/2592067756 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529920503 0 0) 0x7f4e40069820 con 0x7f4e400adb60 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e44816700 1 -- 172.21.15.33:0/2592067756 >> v1:172.21.15.167:6789/0 conn(0x7f4e400adb60 legacy=0x7f4e400c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e44816700 1 -- 172.21.15.33:0/2592067756 >> v1:172.21.15.33:6789/0 conn(0x7f4e400adf30 legacy=0x7f4e40069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e44816700 1 -- 172.21.15.33:0/2592067756 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40068180 con 0x7f4e40067970 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e44816700 1 --2- 172.21.15.33:0/2592067756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e40067970 0x7f4e4015a5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009f20 tx=0x7f4e3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40bb4a428d57cfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e37fff700 1 -- 172.21.15.33:0/2592067756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00e070 con 0x7f4e40067970 2023-10-06T02:46:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e37fff700 1 -- 172.21.15.33:0/2592067756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c00b980 con 0x7f4e40067970 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e37fff700 1 -- 172.21.15.33:0/2592067756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0092e0 con 0x7f4e40067970 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 -- 172.21.15.33:0/2592067756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e40067970 msgr2=0x7f4e4015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 --2- 172.21.15.33:0/2592067756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e40067970 0x7f4e4015a5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009f20 tx=0x7f4e3c009840 comp rx=0 tx=0).stop 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 -- 172.21.15.33:0/2592067756 shutdown_connections 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 --2- 172.21.15.33:0/2592067756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e40067970 0x7f4e4015a5d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 -- 172.21.15.33:0/2592067756 >> 172.21.15.33:0/2592067756 conn(0x7f4e400c8240 msgr2=0x7f4e400c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 -- 172.21.15.33:0/2592067756 shutdown_connections 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.729+0000 7f4e46019700 1 -- 172.21.15.33:0/2592067756 wait complete. 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.733+0000 7f4e46019700 1 Processor -- start 2023-10-06T02:46:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.733+0000 7f4e46019700 1 -- start start 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.733+0000 7f4e46019700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e40067970 0x7f4e400ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.733+0000 7f4e46019700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e400adb60 0x7f4e4014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 0x7f4e40159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c9e70 con 0x7f4e400adf30 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4014f8e0 con 0x7f4e400adb60 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c9540 con 0x7f4e40067970 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 0x7f4e40159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e44816700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e400adb60 0x7f4e4014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45017700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e40067970 0x7f4e400ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 0x7f4e40159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33234/0 (socket says 172.21.15.33:33234) 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 -- 172.21.15.33:0/3461911641 learned_addr learned my addr 172.21.15.33:0/3461911641 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 -- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e40067970 msgr2=0x7f4e400ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e40067970 0x7f4e400ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 -- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e400adb60 msgr2=0x7f4e4014f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e400adb60 0x7f4e4014f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 -- 172.21.15.33:0/3461911641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009ae0 con 0x7f4e400adf30 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e44816700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e400adb60 0x7f4e4014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e45818700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 0x7f4e40159260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800c810 tx=0x7f4e38007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38007300 con 0x7f4e400adf30 2023-10-06T02:46:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e38007480 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400c9820 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38004830 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e401597a0 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e400c9ca0 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.737+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4e38004be0 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.741+0000 7f4e357fa700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4e24060550 0x7f4e24062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.741+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4e3807bd40 con 0x7f4e400adf30 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.741+0000 7f4e45017700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4e24060550 0x7f4e24062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.741+0000 7f4e45017700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4e24060550 0x7f4e24062a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4e4006b4f0 tx=0x7f4e30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.745+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e3804c070 con 0x7f4e400adf30 2023-10-06T02:46:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.949+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e40157eb0 con 0x7f4e400adf30 2023-10-06T02:46:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.949+0000 7f4e357fa700 1 -- 172.21.15.33:0/3461911641 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f4e40157eb0 con 0x7f4e400adf30 2023-10-06T02:46:44.181 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:44.181 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4e24060550 msgr2=0x7f4e24062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4e24060550 0x7f4e24062a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4e4006b4f0 tx=0x7f4e30009040 comp rx=0 tx=0).stop 2023-10-06T02:46:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 msgr2=0x7f4e40159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 0x7f4e40159260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800c810 tx=0x7f4e38007e80 comp rx=0 tx=0).stop 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 shutdown_connections 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e400adf30 0x7f4e40159260 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e400adb60 0x7f4e4014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f4e24060550 0x7f4e24062a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 --2- 172.21.15.33:0/3461911641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e40067970 0x7f4e400ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 >> 172.21.15.33:0/3461911641 conn(0x7f4e400c8240 msgr2=0x7f4e400c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 shutdown_connections 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:43.953+0000 7f4e46019700 1 -- 172.21.15.33:0/3461911641 wait complete. 2023-10-06T02:46:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:44.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2023-10-06T02:46:44.305 INFO:teuthology.orchestra.run.smithi033.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":560,"op_send_bytes":24135,"op_resend":31,"op_reply":560,"oplen_avg":{"avgcount":560,"sum":596},"op":560,"op_r":0,"op_w":560,"op_rmw":0,"op_pg":0,"osdop_stat":1,"osdop_create":14,"osdop_read":0,"osdop_write":9,"osdop_writefull":11,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":506,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":28,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":44,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":24,"omap_rd":0,"omap_del":3}} 2023-10-06T02:46:44.305 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 44, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 0, "omap_wr": 24, "op": 560, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 0, "op_reply": 560, "op_resend": 31, "op_rmw": 0, "op_send": 560, "op_send_bytes": 24135, "op_w": 560, "oplen_avg": { "avgcount": 560, "sum": 596 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 14, "osdop_delete": 506, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 28, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 1, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 9, "osdop_writefull": 11, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2023-10-06T02:46:44.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.409+0000 7f8a817d3700 1 Processor -- start 2023-10-06T02:46:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.409+0000 7f8a817d3700 1 -- start start 2023-10-06T02:46:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a7c158d00 0x7f8a7c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 0x7f8a7c0a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c069f60 con 0x7f8a7c0c7520 2023-10-06T02:46:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c06a240 con 0x7f8a7c158d00 2023-10-06T02:46:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c06a520 con 0x7f8a7c06b000 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 0x7f8a7c0a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 0x7f8a7c0a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55704/0 (socket says 172.21.15.33:55704) 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 -- 172.21.15.33:0/1875137817 learned_addr learned my addr 172.21.15.33:0/1875137817 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a7bfff700 1 --2- 172.21.15.33:0/1875137817 >> v2:172.21.15.167:3300/0 conn(0x7f8a7c158d00 0x7f8a7c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 -- 172.21.15.33:0/1875137817 >> v2:172.21.15.167:3300/0 conn(0x7f8a7c158d00 msgr2=0x7f8a7c0c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 --2- 172.21.15.33:0/1875137817 >> v2:172.21.15.167:3300/0 conn(0x7f8a7c158d00 0x7f8a7c0c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 -- 172.21.15.33:0/1875137817 >> v1:172.21.15.167:6789/0 conn(0x7f8a7c0c7520 legacy=0x7f8a7c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 -- 172.21.15.33:0/1875137817 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c06a800 con 0x7f8a7c06b000 2023-10-06T02:46:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a80fd2700 1 --2- 172.21.15.33:0/1875137817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 0x7f8a7c0a8a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009f20 tx=0x7f8a74009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeee4a5900f194f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a7b7fe700 1 -- 172.21.15.33:0/1875137817 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400e070 con 0x7f8a7c06b000 2023-10-06T02:46:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a7b7fe700 1 -- 172.21.15.33:0/1875137817 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400b8d0 con 0x7f8a7c06b000 2023-10-06T02:46:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a7b7fe700 1 -- 172.21.15.33:0/1875137817 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a740092e0 con 0x7f8a7c06b000 2023-10-06T02:46:44.419 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- 172.21.15.33:0/1875137817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 msgr2=0x7f8a7c0a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 --2- 172.21.15.33:0/1875137817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 0x7f8a7c0a8a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009f20 tx=0x7f8a74009840 comp rx=0 tx=0).stop 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- 172.21.15.33:0/1875137817 shutdown_connections 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 --2- 172.21.15.33:0/1875137817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c06b000 0x7f8a7c0a8a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 --2- 172.21.15.33:0/1875137817 >> v2:172.21.15.167:3300/0 conn(0x7f8a7c158d00 0x7f8a7c0c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- 172.21.15.33:0/1875137817 >> 172.21.15.33:0/1875137817 conn(0x7f8a7c0c8ae0 msgr2=0x7f8a7c0c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- 172.21.15.33:0/1875137817 shutdown_connections 2023-10-06T02:46:44.420 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.413+0000 7f8a817d3700 1 -- 172.21.15.33:0/1875137817 wait complete. 2023-10-06T02:46:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 Processor -- start 2023-10-06T02:46:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 -- start start 2023-10-06T02:46:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a7c06b000 0x7f8a7c0aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 0x7f8a7c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c0ab520 0x7f8a7c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0afdc0 con 0x7f8a7c158d00 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0abaa0 con 0x7f8a7c06b000 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a817d3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0af770 con 0x7f8a7c0ab520 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 0x7f8a7c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 0x7f8a7c15a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33244/0 (socket says 172.21.15.33:33244) 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 -- 172.21.15.33:0/3856669637 learned_addr learned my addr 172.21.15.33:0/3856669637 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a80fd2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c0ab520 0x7f8a7c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a7bfff700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a7c06b000 0x7f8a7c0aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 -- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c0ab520 msgr2=0x7f8a7c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c0ab520 0x7f8a7c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 -- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a7c06b000 msgr2=0x7f8a7c0aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a7c06b000 0x7f8a7c0aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.417+0000 7f8a73fff700 1 -- 172.21.15.33:0/3856669637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74009ae0 con 0x7f8a7c158d00 2023-10-06T02:46:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a73fff700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 0x7f8a7c15a400 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c810 tx=0x7f8a64007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64011590 con 0x7f8a7c158d00 2023-10-06T02:46:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a64011730 con 0x7f8a7c158d00 2023-10-06T02:46:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c0afa50 con 0x7f8a7c158d00 2023-10-06T02:46:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64011af0 con 0x7f8a7c158d00 2023-10-06T02:46:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c0add30 con 0x7f8a7c158d00 2023-10-06T02:46:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c0aff30 con 0x7f8a7c158d00 2023-10-06T02:46:44.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.421+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8a64012420 con 0x7f8a7c158d00 2023-10-06T02:46:44.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.425+0000 7f8a78ff9700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8a5c060550 0x7f8a5c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:44.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.425+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8a6407c7b0 con 0x7f8a7c158d00 2023-10-06T02:46:44.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.425+0000 7f8a7bfff700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8a5c060550 0x7f8a5c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.425+0000 7f8a7bfff700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8a5c060550 0x7f8a5c062a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009db0 tx=0x7f8a6c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:44.435 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.429+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8a6404cb10 con 0x7f8a7c158d00 2023-10-06T02:46:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.629+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a7c071040 con 0x7f8a7c158d00 2023-10-06T02:46:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.629+0000 7f8a78ff9700 1 -- 172.21.15.33:0/3856669637 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f8a64021090 con 0x7f8a7c158d00 2023-10-06T02:46:44.812 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:44.812 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8a5c060550 msgr2=0x7f8a5c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8a5c060550 0x7f8a5c062a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009db0 tx=0x7f8a6c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 msgr2=0x7f8a7c15a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 0x7f8a7c15a400 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c810 tx=0x7f8a64007f30 comp rx=0 tx=0).stop 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 shutdown_connections 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8a5c060550 0x7f8a5c062a00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7c0ab520 0x7f8a7c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a7c158d00 0x7f8a7c15a400 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 --2- 172.21.15.33:0/3856669637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a7c06b000 0x7f8a7c0aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 >> 172.21.15.33:0/3856669637 conn(0x7f8a7c0c8ae0 msgr2=0x7f8a7c0aa030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 shutdown_connections 2023-10-06T02:46:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:44.633+0000 7f8a817d3700 1 -- 172.21.15.33:0/3856669637 wait complete. 2023-10-06T02:46:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:44.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump mds_cache 2023-10-06T02:46:45.015 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0}} 2023-10-06T02:46:45.666 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 } } 2023-10-06T02:46:45.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T02:46:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 Processor -- start 2023-10-06T02:46:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 -- start start 2023-10-06T02:46:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c400adb60 0x7f9c400c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c40067970 0x7f9c4015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c400adf30 0x7f9c4015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c40069820 con 0x7f9c400adf30 2023-10-06T02:46:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.781+0000 7f9c46c16700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c40067ea0 con 0x7f9c40067970 2023-10-06T02:46:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40068180 con 0x7f9c400adb60 2023-10-06T02:46:45.788 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46415700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c400adf30 0x7f9c4015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c40067970 0x7f9c4015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.789 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46415700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9c400adf30 0x7f9c4015ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33254/0 (socket says 172.21.15.33:33254) 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45c14700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c400adb60 0x7f9c400c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46415700 1 -- 172.21.15.33:0/66046763 learned_addr learned my addr 172.21.15.33:0/66046763 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 -- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c400adb60 msgr2=0x7f9c400c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 --2- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c400adb60 0x7f9c400c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 -- 172.21.15.33:0/66046763 >> v2:172.21.15.33:3300/0 conn(0x7f9c400adf30 msgr2=0x7f9c4015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 --2- 172.21.15.33:0/66046763 >> v2:172.21.15.33:3300/0 conn(0x7f9c400adf30 0x7f9c4015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 -- 172.21.15.33:0/66046763 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c40068460 con 0x7f9c40067970 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c45413700 1 --2- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40067970 0x7f9c4015a5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c40068b90 tx=0x7f9c3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808480ecf448993b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c44c12700 1 -- 172.21.15.33:0/66046763 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00e070 con 0x7f9c40067970 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c44c12700 1 -- 172.21.15.33:0/66046763 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00bab0 con 0x7f9c40067970 2023-10-06T02:46:45.790 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c44c12700 1 -- 172.21.15.33:0/66046763 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00be70 con 0x7f9c40067970 2023-10-06T02:46:45.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 -- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40067970 msgr2=0x7f9c4015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:45.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 --2- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40067970 0x7f9c4015a5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c40068b90 tx=0x7f9c3c009780 comp rx=0 tx=0).stop 2023-10-06T02:46:45.791 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 -- 172.21.15.33:0/66046763 shutdown_connections 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 --2- 172.21.15.33:0/66046763 >> v2:172.21.15.33:3300/0 conn(0x7f9c400adf30 0x7f9c4015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 --2- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40067970 0x7f9c4015a5d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 --2- 172.21.15.33:0/66046763 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c400adb60 0x7f9c400c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 -- 172.21.15.33:0/66046763 >> 172.21.15.33:0/66046763 conn(0x7f9c400c8240 msgr2=0x7f9c400c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 -- 172.21.15.33:0/66046763 shutdown_connections 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.785+0000 7f9c46c16700 1 -- 172.21.15.33:0/66046763 wait complete. 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 Processor -- start 2023-10-06T02:46:45.792 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 -- start start 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c40067970 0x7f9c400ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 0x7f9c400aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c400adf30 0x7f9c400af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400af900 con 0x7f9c400adb60 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400c4c70 con 0x7f9c400adf30 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400afba0 con 0x7f9c40067970 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 0x7f9c400aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.794 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 0x7f9c400aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33262/0 (socket says 172.21.15.33:33262) 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 -- 172.21.15.33:0/3864805039 learned_addr learned my addr 172.21.15.33:0/3864805039 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46415700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c400adf30 0x7f9c400af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45c14700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c40067970 0x7f9c400ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 -- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c40067970 msgr2=0x7f9c400ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c40067970 0x7f9c400ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 -- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c400adf30 msgr2=0x7f9c400af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c400adf30 0x7f9c400af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 -- 172.21.15.33:0/3864805039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009a20 con 0x7f9c400adb60 2023-10-06T02:46:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c45413700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 0x7f9c400aee80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b610 tx=0x7f9c3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c012d70 con 0x7f9c400adb60 2023-10-06T02:46:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c027470 con 0x7f9c400adb60 2023-10-06T02:46:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c027830 con 0x7f9c400adb60 2023-10-06T02:46:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c400afe80 con 0x7f9c400adb60 2023-10-06T02:46:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.789+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c400b0ef0 con 0x7f9c400adb60 2023-10-06T02:46:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.793+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c400b0380 con 0x7f9c400adb60 2023-10-06T02:46:45.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.793+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c3c024070 con 0x7f9c400adb60 2023-10-06T02:46:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.793+0000 7f9c35ffb700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9c24060550 0x7f9c24062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.793+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9c3c07f3d0 con 0x7f9c400adb60 2023-10-06T02:46:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.793+0000 7f9c45c14700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9c24060550 0x7f9c24062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.797+0000 7f9c45c14700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9c24060550 0x7f9c24062a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009db0 tx=0x7f9c300091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:45.801+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c3c04f680 con 0x7f9c400adb60 2023-10-06T02:46:46.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.005+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c40071040 con 0x7f9c400adb60 2023-10-06T02:46:46.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.005+0000 7f9c35ffb700 1 -- 172.21.15.33:0/3864805039 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9c3c052ca0 con 0x7f9c400adb60 2023-10-06T02:46:46.011 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9c24060550 msgr2=0x7f9c24062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9c24060550 0x7f9c24062a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009db0 tx=0x7f9c300091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 msgr2=0x7f9c400aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 0x7f9c400aee80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b610 tx=0x7f9c3c019000 comp rx=0 tx=0).stop 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 shutdown_connections 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c400adf30 0x7f9c400af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c400adb60 0x7f9c400aee80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:46.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9c24060550 0x7f9c24062a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:46.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 --2- 172.21.15.33:0/3864805039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c40067970 0x7f9c400ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:46.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 >> 172.21.15.33:0/3864805039 conn(0x7f9c400c8240 msgr2=0x7f9c4006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:46.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 shutdown_connections 2023-10-06T02:46:46.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:46.009+0000 7f9c46c16700 1 -- 172.21.15.33:0/3864805039 wait complete. 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T02:46:46.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:46:46.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:46.026 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2023-10-06T02:46:46.033 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T02:46:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42506sec preferred_lft 42506sec 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:72:46:44:8b brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T02:46:46.035 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 92:7d:32:8e:0c:d8 brd ff:ff:ff:ff:ff:ff 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::5ca2:30ff:feae:fa92/64 scope link 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ee:c8:8e:78:96:20 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ecc8:8eff:fe78:9620/64 scope link 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T02:46:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 92:7d:32:8e:0c:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:46:46.037 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::907d:32ff:fe8e:cd8/64 scope link 2023-10-06T02:46:46.037 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2023-10-06T02:46:46.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:46.037 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T02:46:46.051 INFO:teuthology.orchestra.run.smithi033.stdout:1 2023-10-06T02:46:46.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:46.053 DEBUG:teuthology.orchestra.run.smithi033:> route 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T02:46:46.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:46.062 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2023-10-06T02:46:46.062 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2023-10-06T02:46:46.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T02:46:46.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T02:46:46.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T02:46:46.062 DEBUG:teuthology.orchestra.run.smithi033:> ') 2023-10-06T02:46:46.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:46.098 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:46:46.106 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:46:46.106 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:46:46.106 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:46:46.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:46.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2023-10-06T02:46:46.162 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T02:46:46.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:46:46.162 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:46.168 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T02:46:46.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T02:46:46.169 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:46.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:46:46.234 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:46.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:46.248 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:46.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:46.249 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:46:46.256 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T02:46:46.257 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T02:46:46.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:46:46.273 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:46.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:46.328 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:46.329 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:46.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:46.329 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:46:46.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2023-10-06T02:46:46.337+0000 7fc46e523380 -1 init, newargv = 0x5563b245fd10 newargc=15 2023-10-06T02:46:46.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[44243]: starting ceph client 2023-10-06T02:46:46.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[44243]: starting fuse 2023-10-06T02:46:47.436 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2023-10-06T02:46:47.456 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T02:46:47.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T02:46:47.467 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T02:46:47.468 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T02:46:47.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:47.469 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2023-10-06T02:46:47.518 INFO:teuthology.orchestra.run.smithi033.stdout:52 2023-10-06T02:46:47.519 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-06T02:46:47.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T02:46:47.519 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2023-10-06T02:46:47.519 DEBUG:teuthology.orchestra.run.smithi033:> import re 2023-10-06T02:46:47.519 DEBUG:teuthology.orchestra.run.smithi033:> import os 2023-10-06T02:46:47.519 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2023-10-06T02:46:47.519 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2023-10-06T02:46:47.520 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T02:46:47.521 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2023-10-06T02:46:47.522 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2023-10-06T02:46:47.522 DEBUG:teuthology.orchestra.run.smithi033:> return f 2023-10-06T02:46:47.522 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T02:46:47.522 DEBUG:teuthology.orchestra.run.smithi033:> 2023-10-06T02:46:47.522 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2023-10-06T02:46:47.522 DEBUG:teuthology.orchestra.run.smithi033:> ' 2023-10-06T02:46:47.608 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.44243.asok 2023-10-06T02:46:47.793 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44243.asok 2023-10-06T02:46:47.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:47.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44243.asok status 2023-10-06T02:46:47.894 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:46:47.894 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "44243", 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5448, 2023-10-06T02:46:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5448 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 338459952 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2023-10-06T02:46:47.896 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 338459952 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.5448 192.168.0.1:0/338459952", 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/338459952", 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 16, 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 44, 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2023-10-06T02:46:47.897 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:46:47.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:47.904 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2023-10-06T02:46:47.904 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:47.910 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2023-10-06T02:46:47.910 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:47.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:47.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:47.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T02:46:47.968 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf mydir) 2023-10-06T02:46:47.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:48.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.105+0000 7fef19494700 1 Processor -- start 2023-10-06T02:46:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.105+0000 7fef19494700 1 -- start start 2023-10-06T02:46:48.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158bf0 0x7fef140b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fef140b0530 0x7fef140b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef140b2610 con 0x7fef14158fc0 2023-10-06T02:46:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef14067c10 con 0x7fef140b0530 2023-10-06T02:46:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1415d830 con 0x7fef14158bf0 2023-10-06T02:46:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef137fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fef14158fc0 0x7fef1415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51968/0 (socket says 172.21.15.33:51968) 2023-10-06T02:46:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef137fe700 1 -- 172.21.15.33:0/2673374731 learned_addr learned my addr 172.21.15.33:0/2673374731 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef12ffd700 1 --2- 172.21.15.33:0/2673374731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158bf0 0x7fef140b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef127fc700 1 --2- 172.21.15.33:0/2673374731 >> v2:172.21.15.167:3300/0 conn(0x7fef140b0530 0x7fef140b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef11ffb700 1 -- 172.21.15.33:0/2673374731 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807809146 0 0) 0x7fef0c001490 con 0x7fef14158fc0 2023-10-06T02:46:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef11ffb700 1 -- 172.21.15.33:0/2673374731 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feefc003570 con 0x7fef14158fc0 2023-10-06T02:46:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef12ffd700 1 -- 172.21.15.33:0/2673374731 >> v2:172.21.15.167:3300/0 conn(0x7fef140b0530 msgr2=0x7fef140b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef12ffd700 1 --2- 172.21.15.33:0/2673374731 >> v2:172.21.15.167:3300/0 conn(0x7fef140b0530 0x7fef140b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef12ffd700 1 -- 172.21.15.33:0/2673374731 >> v1:172.21.15.33:6789/0 conn(0x7fef14158fc0 legacy=0x7fef1415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef12ffd700 1 -- 172.21.15.33:0/2673374731 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140b00a0 con 0x7fef14158bf0 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef127fc700 1 --2- 172.21.15.33:0/2673374731 >> v2:172.21.15.167:3300/0 conn(0x7fef140b0530 0x7fef140b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef12ffd700 1 --2- 172.21.15.33:0/2673374731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158bf0 0x7fef140b1b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef080098e0 tx=0x7fef080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49392cf6ec8ab61b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef11ffb700 1 -- 172.21.15.33:0/2673374731 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08011630 con 0x7fef14158bf0 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef11ffb700 1 -- 172.21.15.33:0/2673374731 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08011790 con 0x7fef14158bf0 2023-10-06T02:46:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef11ffb700 1 -- 172.21.15.33:0/2673374731 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08011b10 con 0x7fef14158bf0 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- 172.21.15.33:0/2673374731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158bf0 msgr2=0x7fef140b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 --2- 172.21.15.33:0/2673374731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158bf0 0x7fef140b1b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef080098e0 tx=0x7fef080091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- 172.21.15.33:0/2673374731 shutdown_connections 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 --2- 172.21.15.33:0/2673374731 >> v2:172.21.15.167:3300/0 conn(0x7fef140b0530 0x7fef140b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 --2- 172.21.15.33:0/2673374731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158bf0 0x7fef140b1b90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- 172.21.15.33:0/2673374731 >> 172.21.15.33:0/2673374731 conn(0x7fef140cfd70 msgr2=0x7fef140d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- 172.21.15.33:0/2673374731 shutdown_connections 2023-10-06T02:46:48.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.109+0000 7fef19494700 1 -- 172.21.15.33:0/2673374731 wait complete. 2023-10-06T02:46:48.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 Processor -- start 2023-10-06T02:46:48.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 -- start start 2023-10-06T02:46:48.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef14158bf0 0x7fef141500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 0x7fef140c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 0x7fef14159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140c4730 con 0x7fef140c58b0 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140c5e80 con 0x7fef14158bf0 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef19494700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef140c6160 con 0x7fef14158fc0 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 0x7fef14159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef127fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 0x7fef140c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 0x7fef14159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33272/0 (socket says 172.21.15.33:33272) 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef127fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 0x7fef140c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:55748/0 (socket says 172.21.15.33:55748) 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 -- 172.21.15.33:0/3074516592 learned_addr learned my addr 172.21.15.33:0/3074516592 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef12ffd700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef14158bf0 0x7fef141500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 -- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 msgr2=0x7fef140c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 0x7fef140c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 -- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef14158bf0 msgr2=0x7fef141500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef14158bf0 0x7fef141500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef137fe700 1 -- 172.21.15.33:0/3074516592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04009420 con 0x7fef140c58b0 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.113+0000 7fef127fc700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 0x7fef140c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7fef137fe700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 0x7fef14159460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00cd00 tx=0x7fef0c001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00ce00 con 0x7fef140c58b0 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c00aac0 con 0x7fef140c58b0 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef08009420 con 0x7fef140c58b0 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00ae40 con 0x7fef140c58b0 2023-10-06T02:46:48.121 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef140b2b20 con 0x7fef140c58b0 2023-10-06T02:46:48.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef140710b0 con 0x7fef140c58b0 2023-10-06T02:46:48.122 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fef0c007300 con 0x7fef140c58b0 2023-10-06T02:46:48.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7feefaffd700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fef00060550 0x7fef00062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.117+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fef0c07ccc0 con 0x7fef140c58b0 2023-10-06T02:46:48.123 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.121+0000 7fef12ffd700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fef00060550 0x7fef00062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.121+0000 7fef12ffd700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fef00060550 0x7fef00062a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef080091e0 tx=0x7fef08017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:48.133 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.129+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fef0c04d010 con 0x7fef140c58b0 2023-10-06T02:46:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.325+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef140b2610 con 0x7fef140c58b0 2023-10-06T02:46:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7feefaffd700 1 -- 172.21.15.33:0/3074516592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fef140b2610 con 0x7fef140c58b0 2023-10-06T02:46:48.332 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:48.332 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:48.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fef00060550 msgr2=0x7fef00062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fef00060550 0x7fef00062a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef080091e0 tx=0x7fef08017000 comp rx=0 tx=0).stop 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 msgr2=0x7fef14159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 0x7fef14159460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00cd00 tx=0x7fef0c001ad0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 shutdown_connections 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fef00060550 0x7fef00062a00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef14158fc0 0x7fef140c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef140c58b0 0x7fef14159460 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 --2- 172.21.15.33:0/3074516592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef14158bf0 0x7fef141500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 >> 172.21.15.33:0/3074516592 conn(0x7fef140cfd70 msgr2=0x7fef140ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 shutdown_connections 2023-10-06T02:46:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.329+0000 7fef19494700 1 -- 172.21.15.33:0/3074516592 wait complete. 2023-10-06T02:46:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:48.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-10-06T02:46:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:{"message":"","return_code":0} 2023-10-06T02:46:48.477 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T02:46:48.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 Processor -- start 2023-10-06T02:46:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 -- start start 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f231c067970 0x7f231c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231c069820 con 0x7f231c0adf30 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c067ea0 con 0x7f231c067970 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f232279a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c068180 con 0x7f231c0adb60 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f231c067970 0x7f231c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f231c067970 0x7f231c15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33288/0 (socket says 172.21.15.33:33288) 2023-10-06T02:46:48.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 -- 172.21.15.33:0/3326559279 learned_addr learned my addr 172.21.15.33:0/3326559279 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2321798700 1 --2- 172.21.15.33:0/3326559279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 -- 172.21.15.33:0/3326559279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 msgr2=0x7f231c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 --2- 172.21.15.33:0/3326559279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 -- 172.21.15.33:0/3326559279 >> v1:172.21.15.167:6789/0 conn(0x7f231c0adf30 legacy=0x7f231c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 -- 172.21.15.33:0/3326559279 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c068460 con 0x7f231c067970 2023-10-06T02:46:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.581+0000 7f2320f97700 1 --2- 172.21.15.33:0/3326559279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c067970 0x7f231c15a5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f230c0099b0 tx=0x7f230c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f587619d43c4b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f2313fff700 1 -- 172.21.15.33:0/3326559279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c009ea0 con 0x7f231c067970 2023-10-06T02:46:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f2313fff700 1 -- 172.21.15.33:0/3326559279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c00b860 con 0x7f231c067970 2023-10-06T02:46:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f2313fff700 1 -- 172.21.15.33:0/3326559279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c00bc00 con 0x7f231c067970 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 -- 172.21.15.33:0/3326559279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c067970 msgr2=0x7f231c15a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 --2- 172.21.15.33:0/3326559279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c067970 0x7f231c15a5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f230c0099b0 tx=0x7f230c009250 comp rx=0 tx=0).stop 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 -- 172.21.15.33:0/3326559279 shutdown_connections 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 --2- 172.21.15.33:0/3326559279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c067970 0x7f231c15a5d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 --2- 172.21.15.33:0/3326559279 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 -- 172.21.15.33:0/3326559279 >> 172.21.15.33:0/3326559279 conn(0x7f231c0c8240 msgr2=0x7f231c0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 -- 172.21.15.33:0/3326559279 shutdown_connections 2023-10-06T02:46:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 -- 172.21.15.33:0/3326559279 wait complete. 2023-10-06T02:46:48.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 Processor -- start 2023-10-06T02:46:48.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.585+0000 7f232279a700 1 -- start start 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f231c067970 0x7f231c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c168d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c0adf30 0x7f231c1692d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1681c0 con 0x7f231c0adf30 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1423f0 con 0x7f231c067970 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1420c0 con 0x7f231c0adb60 2023-10-06T02:46:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c0adf30 0x7f231c1692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321798700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f231c067970 0x7f231c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321798700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f231c067970 0x7f231c152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:52128/0 (socket says 172.21.15.33:52128) 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2320f97700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c168d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321798700 1 -- 172.21.15.33:0/2740276105 learned_addr learned my addr 172.21.15.33:0/2740276105 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 -- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 msgr2=0x7f231c168d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c168d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 -- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f231c067970 msgr2=0x7f231c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f231c067970 0x7f231c152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 -- 172.21.15.33:0/2740276105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c0094f0 con 0x7f231c0adf30 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f2321f99700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c0adf30 0x7f231c1692d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ae390 tx=0x7f23140015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314011590 con 0x7f231c0adf30 2023-10-06T02:46:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2314011710 con 0x7f231c0adf30 2023-10-06T02:46:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c169950 con 0x7f231c0adf30 2023-10-06T02:46:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c169f10 con 0x7f231c0adf30 2023-10-06T02:46:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.589+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314011ad0 con 0x7f231c0adf30 2023-10-06T02:46:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.593+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c142560 con 0x7f231c0adf30 2023-10-06T02:46:48.597 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.593+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2314012420 con 0x7f231c0adf30 2023-10-06T02:46:48.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.593+0000 7f23117fa700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2300060550 0x7f2300062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:48.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.593+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f231407c810 con 0x7f231c0adf30 2023-10-06T02:46:48.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.593+0000 7f2321798700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2300060550 0x7f2300062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.593+0000 7f2321798700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2300060550 0x7f2300062a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f231c06d650 tx=0x7f23180091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:48.606 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.601+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f231404cac0 con 0x7f231c0adf30 2023-10-06T02:46:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.793+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f231c071040 con 0x7f231c0adf30 2023-10-06T02:46:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.793+0000 7f23117fa700 1 -- 172.21.15.33:0/2740276105 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f23140500e0 con 0x7f231c0adf30 2023-10-06T02:46:48.798 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:48.798 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2300060550 msgr2=0x7f2300062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2300060550 0x7f2300062a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f231c06d650 tx=0x7f23180091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c0adf30 msgr2=0x7f231c1692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c0adf30 0x7f231c1692d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ae390 tx=0x7f23140015f0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 shutdown_connections 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2300060550 0x7f2300062a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f231c0adb60 0x7f231c168d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231c0adf30 0x7f231c1692d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 --2- 172.21.15.33:0/2740276105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f231c067970 0x7f231c152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 >> 172.21.15.33:0/2740276105 conn(0x7f231c0c8240 msgr2=0x7f231c1540b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 shutdown_connections 2023-10-06T02:46:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:48.797+0000 7f232279a700 1 -- 172.21.15.33:0/2740276105 wait complete. 2023-10-06T02:46:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:48.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump mds_cache 2023-10-06T02:46:48.921 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_cache":{"num_strays":2,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":2,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0}} 2023-10-06T02:46:48.921 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 2, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 2, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 } } 2023-10-06T02:46:48.922 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2023-10-06T02:46:49.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:46:49.882 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:46:53.923 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:54.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 Processor -- start 2023-10-06T02:46:54.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 -- start start 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a40ade00 0x7fe6a40c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6a40c5940 0x7fe6a40c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6a4066fb0 0x7fe6a40c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40c80c0 con 0x7fe6a4066fb0 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a406add0 con 0x7fe6a40c5940 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6aa7f2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c7c40 con 0x7fe6a40ade00 2023-10-06T02:46:54.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6a4066fb0 0x7fe6a40c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe6a4066fb0 0x7fe6a40c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59152/0 (socket says 172.21.15.33:59152) 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 -- 172.21.15.33:0/4168162342 learned_addr learned my addr 172.21.15.33:0/4168162342 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a8fef700 1 --2- 172.21.15.33:0/4168162342 >> v2:172.21.15.167:3300/0 conn(0x7fe6a40c5940 0x7fe6a40c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a97f0700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a40ade00 0x7fe6a40c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 -- 172.21.15.33:0/4168162342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a40ade00 msgr2=0x7fe6a40c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a40ade00 0x7fe6a40c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 -- 172.21.15.33:0/4168162342 >> v2:172.21.15.167:3300/0 conn(0x7fe6a40c5940 msgr2=0x7fe6a40c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 --2- 172.21.15.33:0/4168162342 >> v2:172.21.15.167:3300/0 conn(0x7fe6a40c5940 0x7fe6a40c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 -- 172.21.15.33:0/4168162342 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40ae1d0 con 0x7fe6a4066fb0 2023-10-06T02:46:54.045 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a97f0700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a40ade00 0x7fe6a40c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:54.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.037+0000 7fe6a9ff1700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a4066fb0 0x7fe6a40c9550 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40692e0 tx=0x7fe69c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a887734a073019a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe69bfff700 1 -- 172.21.15.33:0/4168162342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00e070 con 0x7fe6a4066fb0 2023-10-06T02:46:54.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe69bfff700 1 -- 172.21.15.33:0/4168162342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00ba40 con 0x7fe6a4066fb0 2023-10-06T02:46:54.046 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe69bfff700 1 -- 172.21.15.33:0/4168162342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0092e0 con 0x7fe6a4066fb0 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/4168162342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a4066fb0 msgr2=0x7fe6a40c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a4066fb0 0x7fe6a40c9550 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40692e0 tx=0x7fe69c009840 comp rx=0 tx=0).stop 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/4168162342 shutdown_connections 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a4066fb0 0x7fe6a40c9550 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/4168162342 >> v2:172.21.15.167:3300/0 conn(0x7fe6a40c5940 0x7fe6a40c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/4168162342 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a40ade00 0x7fe6a40c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/4168162342 >> 172.21.15.33:0/4168162342 conn(0x7fe6a40c8240 msgr2=0x7fe6a40c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/4168162342 shutdown_connections 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/4168162342 wait complete. 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 Processor -- start 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- start start 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a4066fb0 0x7fe6a4155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a40ade00 0x7fe6a40c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a40c5940 0x7fe6a4146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c4fb0 con 0x7fe6a40ade00 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a414bc30 con 0x7fe6a40c5940 2023-10-06T02:46:54.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6aa7f2700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4156440 con 0x7fe6a4066fb0 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6a8fef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a40ade00 0x7fe6a40c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6a97f0700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a4066fb0 0x7fe6a4155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6a97f0700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a4066fb0 0x7fe6a4155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35448/0 (socket says 172.21.15.33:35448) 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6a97f0700 1 -- 172.21.15.33:0/2799044400 learned_addr learned my addr 172.21.15.33:0/2799044400 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.041+0000 7fe6a9ff1700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a40c5940 0x7fe6a4146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6a8fef700 1 -- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a4066fb0 msgr2=0x7fe6a4155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6a8fef700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a4066fb0 0x7fe6a4155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6a8fef700 1 -- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a40c5940 msgr2=0x7fe6a4146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6a8fef700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a40c5940 0x7fe6a4146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6a8fef700 1 -- 172.21.15.33:0/2799044400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009ae0 con 0x7fe6a40ade00 2023-10-06T02:46:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6a8fef700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a40ade00 0x7fe6a40c43d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe69400d8f0 tx=0x7fe694008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6940125f0 con 0x7fe6a40ade00 2023-10-06T02:46:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe694012750 con 0x7fe6a40ade00 2023-10-06T02:46:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694012af0 con 0x7fe6a40ade00 2023-10-06T02:46:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a40c4cd0 con 0x7fe6a40ade00 2023-10-06T02:46:54.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a414b8c0 con 0x7fe6a40ade00 2023-10-06T02:46:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a414bdc0 con 0x7fe6a40ade00 2023-10-06T02:46:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.045+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe694013480 con 0x7fe6a40ade00 2023-10-06T02:46:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.049+0000 7fe6997fa700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe688060580 0x7fe688062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.049+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe69407ccb0 con 0x7fe6a40ade00 2023-10-06T02:46:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.049+0000 7fe6a97f0700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe688060580 0x7fe688062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.049+0000 7fe6a97f0700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe688060580 0x7fe688062a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.057+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe69404d000 con 0x7fe6a40ade00 2023-10-06T02:46:54.259 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.253+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6a4071040 con 0x7fe6a40ade00 2023-10-06T02:46:54.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6997fa700 1 -- 172.21.15.33:0/2799044400 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fe694050620 con 0x7fe6a40ade00 2023-10-06T02:46:54.260 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:54.260 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe688060580 msgr2=0x7fe688062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe688060580 0x7fe688062a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0008040 comp rx=0 tx=0).stop 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a40ade00 msgr2=0x7fe6a40c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a40ade00 0x7fe6a40c43d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe69400d8f0 tx=0x7fe694008040 comp rx=0 tx=0).stop 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 shutdown_connections 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a40c5940 0x7fe6a4146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a40ade00 0x7fe6a40c43d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe688060580 0x7fe688062a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 --2- 172.21.15.33:0/2799044400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6a4066fb0 0x7fe6a4155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 >> 172.21.15.33:0/2799044400 conn(0x7fe6a40c8240 msgr2=0x7fe6a40c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 shutdown_connections 2023-10-06T02:46:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.257+0000 7fe6aa7f2700 1 -- 172.21.15.33:0/2799044400 wait complete. 2023-10-06T02:46:54.264 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:54.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump mds_cache 2023-10-06T02:46:54.387 INFO:teuthology.orchestra.run.smithi033.stdout:{"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":2,"strays_enqueued":2,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0}} 2023-10-06T02:46:54.387 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 2, "strays_enqueued": 2, "strays_migrated": 0, "strays_reintegrated": 0 } } 2023-10-06T02:46:54.387 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-06T02:46:54.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.485+0000 7f58dec21700 1 Processor -- start 2023-10-06T02:46:54.491 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.485+0000 7f58dec21700 1 -- start start 2023-10-06T02:46:54.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f58d8155350 0x7f58d80c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80c86b0 con 0x7f58d806e170 2023-10-06T02:46:54.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80c7aa0 con 0x7f58d806e540 2023-10-06T02:46:54.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d8068080 con 0x7f58d8155350 2023-10-06T02:46:54.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dc9bd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f58d806e170 0x7f58d8067970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57274/0 (socket says 172.21.15.33:57274) 2023-10-06T02:46:54.493 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dc9bd700 1 -- 172.21.15.33:0/3455198949 learned_addr learned my addr 172.21.15.33:0/3455198949 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dd1be700 1 --2- 172.21.15.33:0/3455198949 >> v2:172.21.15.167:3300/0 conn(0x7f58d8155350 0x7f58d80c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58d77fe700 1 -- 172.21.15.33:0/3455198949 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938679663 0 0) 0x7f58d80c86b0 con 0x7f58d806e170 2023-10-06T02:46:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58d77fe700 1 -- 172.21.15.33:0/3455198949 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c8003570 con 0x7f58d806e170 2023-10-06T02:46:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dd1be700 1 -- 172.21.15.33:0/3455198949 >> v1:172.21.15.167:6789/0 conn(0x7f58d806e540 legacy=0x7f58d8154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dd1be700 1 -- 172.21.15.33:0/3455198949 >> v1:172.21.15.33:6789/0 conn(0x7f58d806e170 legacy=0x7f58d8067970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dd1be700 1 -- 172.21.15.33:0/3455198949 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80c7d80 con 0x7f58d8155350 2023-10-06T02:46:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dd1be700 1 --2- 172.21.15.33:0/3455198949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d80c7450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e90 tx=0x7f58d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cf08f4d57007f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58d77fe700 1 -- 172.21.15.33:0/3455198949 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000e070 con 0x7f58d8155350 2023-10-06T02:46:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58d77fe700 1 -- 172.21.15.33:0/3455198949 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d000bc40 con 0x7f58d8155350 2023-10-06T02:46:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58d77fe700 1 -- 172.21.15.33:0/3455198949 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0012430 con 0x7f58d8155350 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 -- 172.21.15.33:0/3455198949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 msgr2=0x7f58d80c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 --2- 172.21.15.33:0/3455198949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d80c7450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009e90 tx=0x7f58d0009780 comp rx=0 tx=0).stop 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 -- 172.21.15.33:0/3455198949 shutdown_connections 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 --2- 172.21.15.33:0/3455198949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d80c7450 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.489+0000 7f58dec21700 1 -- 172.21.15.33:0/3455198949 >> 172.21.15.33:0/3455198949 conn(0x7f58d806aa50 msgr2=0x7f58d80cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.493+0000 7f58dec21700 1 -- 172.21.15.33:0/3455198949 shutdown_connections 2023-10-06T02:46:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.493+0000 7f58dec21700 1 -- 172.21.15.33:0/3455198949 wait complete. 2023-10-06T02:46:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.493+0000 7f58dec21700 1 Processor -- start 2023-10-06T02:46:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.493+0000 7f58dec21700 1 -- start start 2023-10-06T02:46:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 0x7f58d81251c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58d806e540 0x7f58d80b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81257a0 con 0x7f58d806e170 2023-10-06T02:46:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8125a80 con 0x7f58d8155350 2023-10-06T02:46:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8125d60 con 0x7f58d806e540 2023-10-06T02:46:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 0x7f58d81251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 0x7f58d81251c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59170/0 (socket says 172.21.15.33:59170) 2023-10-06T02:46:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 -- 172.21.15.33:0/181814327 learned_addr learned my addr 172.21.15.33:0/181814327 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dd1be700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58d7fff700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58d806e540 0x7f58d80b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 -- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58d806e540 msgr2=0x7f58d80b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58d806e540 0x7f58d80b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 -- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 msgr2=0x7f58d813af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d813af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 -- 172.21.15.33:0/181814327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d0009a20 con 0x7f58d806e170 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dc9bd700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 0x7f58d81251c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c590 tx=0x7f58cc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc005b30 con 0x7f58d806e170 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc005cb0 con 0x7f58d806e170 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d8126040 con 0x7f58d806e170 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc007320 con 0x7f58d806e170 2023-10-06T02:46:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d80b19e0 con 0x7f58d806e170 2023-10-06T02:46:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.497+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d8126520 con 0x7f58d806e170 2023-10-06T02:46:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.501+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f58cc011470 con 0x7f58d806e170 2023-10-06T02:46:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.501+0000 7f58d4ff9700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f58bc060550 0x7f58bc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.501+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f58cc07e830 con 0x7f58d806e170 2023-10-06T02:46:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.501+0000 7f58d7fff700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f58bc060550 0x7f58bc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.501+0000 7f58d7fff700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f58bc060550 0x7f58bc062a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58d80ca1e0 tx=0x7f58c4007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.505+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f58cc04eb80 con 0x7f58d806e170 2023-10-06T02:46:54.708 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.705+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58d80710b0 con 0x7f58d806e170 2023-10-06T02:46:54.709 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.705+0000 7f58d4ff9700 1 -- 172.21.15.33:0/181814327 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f58cc0521a0 con 0x7f58d806e170 2023-10-06T02:46:54.709 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:54.709 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.705+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f58bc060550 msgr2=0x7f58bc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.705+0000 7f58dec21700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f58bc060550 0x7f58bc062a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58d80ca1e0 tx=0x7f58c4007ea0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.705+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 msgr2=0x7f58d81251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 0x7f58d81251c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c590 tx=0x7f58cc007e00 comp rx=0 tx=0).stop 2023-10-06T02:46:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 shutdown_connections 2023-10-06T02:46:54.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f58bc060550 0x7f58bc062a00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58d806e540 0x7f58d80b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58d8155350 0x7f58d813af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 --2- 172.21.15.33:0/181814327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d806e170 0x7f58d81251c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 >> 172.21.15.33:0/181814327 conn(0x7f58d806aa50 msgr2=0x7f58d80cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 shutdown_connections 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.709+0000 7f58dec21700 1 -- 172.21.15.33:0/181814327 wait complete. 2023-10-06T02:46:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:54.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump objecter 2023-10-06T02:46:54.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":592,"op_send_bytes":39574,"op_resend":31,"op_reply":592,"oplen_avg":{"avgcount":592,"sum":643},"op":592,"op_r":2,"op_w":590,"op_rmw":0,"op_pg":0,"osdop_stat":4,"osdop_create":17,"osdop_read":2,"osdop_write":16,"osdop_writefull":17,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":514,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":34,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":44,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":34,"omap_rd":0,"omap_del":5}} 2023-10-06T02:46:54.835 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 44, "map_full": 0, "map_inc": 1, "omap_del": 5, "omap_rd": 0, "omap_wr": 34, "op": 592, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 2, "op_reply": 592, "op_resend": 31, "op_rmw": 0, "op_send": 592, "op_send_bytes": 39574, "op_w": 590, "oplen_avg": { "avgcount": 592, "sum": 643 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 17, "osdop_delete": 514, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 2, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 34, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 4, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 16, "osdop_writefull": 17, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2023-10-06T02:46:54.836 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T02:46:54.836 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2023-10-06T02:46:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 Processor -- start 2023-10-06T02:46:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 -- start start 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c97c66a0 0x5639c97c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 0x5639c97c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c9853730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c96e3840 con 0x5639c97e6650 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c96e34e0 con 0x5639c97c66a0 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522dbd1e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639c96da270 con 0x5639c9870480 2023-10-06T02:46:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 0x5639c97c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 0x5639c97c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59176/0 (socket says 172.21.15.33:59176) 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 -- 172.21.15.33:0/2586721303 learned_addr learned my addr 172.21.15.33:0/2586721303 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522bf09700 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c9853730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522b708700 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c97c66a0 0x5639c97c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 -- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 msgr2=0x5639c9853730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c9853730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 -- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c97c66a0 msgr2=0x5639c97c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c97c66a0 0x5639c97c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 -- 172.21.15.33:0/2586721303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c005db0 con 0x5639c97e6650 2023-10-06T02:46:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f522af07700 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 0x5639c97c8520 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f521400a6e0 tx=0x7f5214007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4548e2b8534b4f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.865+0000 7f5229f05700 1 -- 172.21.15.33:0/2586721303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140115f0 con 0x5639c97e6650 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f5229f05700 1 -- 172.21.15.33:0/2586721303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214011750 con 0x5639c97e6650 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f5229f05700 1 -- 172.21.15.33:0/2586721303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214011af0 con 0x5639c97e6650 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- 172.21.15.33:0/2586721303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 msgr2=0x5639c97c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 0x5639c97c8520 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f521400a6e0 tx=0x7f5214007040 comp rx=0 tx=0).stop 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- 172.21.15.33:0/2586721303 shutdown_connections 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c9853730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97e6650 0x5639c97c8520 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/2586721303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c97c66a0 0x5639c97c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- 172.21.15.33:0/2586721303 >> 172.21.15.33:0/2586721303 conn(0x5639c97e7200 msgr2=0x5639c97e3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- 172.21.15.33:0/2586721303 shutdown_connections 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- 172.21.15.33:0/2586721303 wait complete. 2023-10-06T02:46:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 Processor -- start 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- start start 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97c66a0 0x5639c986a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c986eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c986f070 0x5639c97e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c986fb40 con 0x5639c97c66a0 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c98824b0 con 0x5639c986f070 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522dbd1e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639c986f580 con 0x5639c9870480 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97c66a0 0x5639c986a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97c66a0 0x5639c986a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59184/0 (socket says 172.21.15.33:59184) 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 -- 172.21.15.33:0/7723772 learned_addr learned my addr 172.21.15.33:0/7723772 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522af07700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c986eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522bf09700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c986f070 0x5639c97e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 -- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 msgr2=0x5639c986eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5639c9870480 0x5639c986eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 -- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c986f070 msgr2=0x5639c97e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639c986f070 0x5639c97e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 -- 172.21.15.33:0/7723772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521400d040 con 0x5639c97c66a0 2023-10-06T02:46:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f522b708700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97c66a0 0x5639c986a600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f522400b290 tx=0x7f5224000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.869+0000 7f5223fff700 1 -- 172.21.15.33:0/7723772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400ba90 con 0x5639c97c66a0 2023-10-06T02:46:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5639c96f2250 con 0x5639c97c66a0 2023-10-06T02:46:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f5223fff700 1 -- 172.21.15.33:0/7723772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522400bbf0 con 0x5639c97c66a0 2023-10-06T02:46:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f5223fff700 1 -- 172.21.15.33:0/7723772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224010430 con 0x5639c97c66a0 2023-10-06T02:46:54.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639c96e5770 con 0x5639c97c66a0 2023-10-06T02:46:54.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f5223fff700 1 -- 172.21.15.33:0/7723772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f52240107a0 con 0x5639c97c66a0 2023-10-06T02:46:54.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f5223fff700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f521005eeb0 0x7f5210061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.879 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f5223fff700 1 -- 172.21.15.33:0/7723772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f5224008030 con 0x5639c97c66a0 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5639c98637e0 0x5639c9863bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5639c9882940 con 0x5639c98637e0 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f522af07700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f521005eeb0 0x7f5210061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.873+0000 7f522bf09700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5639c98637e0 0x5639c9863bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522af07700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f521005eeb0 0x7f5210061360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5639c97e6040 tx=0x7f5214019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522bf09700 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5639c98637e0 0x5639c9863bf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522bf09700 1 -- 172.21.15.33:0/7723772 <== osd.0 v2:172.21.15.33:6816/33421 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f521c008ae0 con 0x5639c98637e0 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:error getting omap key set cephfs_metadata/10000000000.00000000: (2) No such file or directory 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5639c98637e0 msgr2=0x5639c9863bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x5639c98637e0 0x5639c9863bf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f521005eeb0 msgr2=0x7f5210061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f521005eeb0 0x7f5210061360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5639c97e6040 tx=0x7f5214019000 comp rx=0 tx=0).stop 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97c66a0 msgr2=0x5639c986a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 --2- 172.21.15.33:0/7723772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639c97c66a0 0x5639c986a600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f522400b290 tx=0x7f5224000bc0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522bf09700 1 -- 172.21.15.33:0/7723772 reap_dead start 2023-10-06T02:46:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 shutdown_connections 2023-10-06T02:46:54.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 >> 172.21.15.33:0/7723772 conn(0x5639c97e7200 msgr2=0x5639c97e3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 shutdown_connections 2023-10-06T02:46:54.883 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.877+0000 7f522dbd1e80 1 -- 172.21.15.33:0/7723772 wait complete. 2023-10-06T02:46:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:54.884 ERROR:tasks.cephfs.filesystem:Command failed on smithi033 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000' 2023-10-06T02:46:54.884 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-10-06T02:46:54.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.913+0000 7f82c00f7e80 1 Processor -- start 2023-10-06T02:46:54.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.913+0000 7f82c00f7e80 1 -- start start 2023-10-06T02:46:54.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 0x55c871eb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871eb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e269a0 0x55c871e23b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c871d24840 con 0x55c871e081b0 2023-10-06T02:46:54.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c871d244e0 con 0x55c871e269a0 2023-10-06T02:46:54.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c871d1b270 con 0x55c871e08580 2023-10-06T02:46:54.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 0x55c871eb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bd42d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871eb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82be42f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e269a0 0x55c871e23b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 0x55c871eb1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59194/0 (socket says 172.21.15.33:59194) 2023-10-06T02:46:54.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bd42d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871eb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35492/0 (socket says 172.21.15.33:35492) 2023-10-06T02:46:54.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 -- 172.21.15.33:0/3179170145 learned_addr learned my addr 172.21.15.33:0/3179170145 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 -- 172.21.15.33:0/3179170145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 msgr2=0x55c871eb1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871eb1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 -- 172.21.15.33:0/3179170145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e269a0 msgr2=0x55c871e23b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e269a0 0x55c871e23b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 -- 172.21.15.33:0/3179170145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8005db0 con 0x55c871e081b0 2023-10-06T02:46:54.923 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82bdc2e700 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 0x55c871eb1140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f82b8001630 tx=0x7f82b8006ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c09e6bdcbf120c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82b7fff700 1 -- 172.21.15.33:0/3179170145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8004150 con 0x55c871e081b0 2023-10-06T02:46:54.926 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82b7fff700 1 -- 172.21.15.33:0/3179170145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8013460 con 0x55c871e081b0 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82b7fff700 1 -- 172.21.15.33:0/3179170145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8013800 con 0x55c871e081b0 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3179170145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 msgr2=0x55c871eb1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 0x55c871eb1140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f82b8001630 tx=0x7f82b8006ba0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3179170145 shutdown_connections 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871eb1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e269a0 0x55c871e23b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3179170145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e081b0 0x55c871eb1140 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3179170145 >> 172.21.15.33:0/3179170145 conn(0x55c871dd70f0 msgr2=0x55c871dda410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3179170145 shutdown_connections 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3179170145 wait complete. 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.917+0000 7f82c00f7e80 1 Processor -- start 2023-10-06T02:46:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 -- start start 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e081b0 0x55c871e1d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871e23ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e269a0 0x55c871ebb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c871ebad30 con 0x55c871e269a0 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c871e1dbb0 con 0x55c871e081b0 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c871e21160 con 0x55c871e08580 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e269a0 0x55c871ebb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e269a0 0x55c871ebb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59206/0 (socket says 172.21.15.33:59206) 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 -- 172.21.15.33:0/3084469957 learned_addr learned my addr 172.21.15.33:0/3084469957 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82bd42d700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871e23ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.928 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82bdc2e700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e081b0 0x55c871e1d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 -- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 msgr2=0x55c871e23ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55c871e08580 0x55c871e23ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 -- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e081b0 msgr2=0x55c871e1d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c871e081b0 0x55c871e1d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 -- 172.21.15.33:0/3084469957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c871d263e0 con 0x55c871e269a0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82be42f700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e269a0 0x55c871ebb1c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82b0008a70 tx=0x7f82b000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82b67fc700 1 -- 172.21.15.33:0/3084469957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000f040 con 0x55c871e269a0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82b67fc700 1 -- 172.21.15.33:0/3084469957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b00108b0 con 0x55c871e269a0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c871d33250 con 0x55c871e269a0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82b67fc700 1 -- 172.21.15.33:0/3084469957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0010c70 con 0x55c871e269a0 2023-10-06T02:46:54.929 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.921+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c871d26770 con 0x55c871e269a0 2023-10-06T02:46:54.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82b67fc700 1 -- 172.21.15.33:0/3084469957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f82b0015660 con 0x55c871e269a0 2023-10-06T02:46:54.930 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82b67fc700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f82a405eeb0 0x7f82a4061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82b67fc700 1 -- 172.21.15.33:0/3084469957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f82b007c640 con 0x55c871e269a0 2023-10-06T02:46:54.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82bdc2e700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f82a405eeb0 0x7f82a4061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c871e241e0 0x55c871ec36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55c871e21ba0 con 0x55c871e241e0 2023-10-06T02:46:54.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82bd42d700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c871e241e0 0x55c871ec36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.931 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.925+0000 7f82bdc2e700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f82a405eeb0 0x7f82a4061360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f82b8001370 tx=0x7f82b8013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82bd42d700 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c871e241e0 0x55c871ec36f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82bd42d700 1 -- 172.21.15.33:0/3084469957 <== osd.6 v2:172.21.15.167:6816/33656 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f82a80089d0 con 0x55c871e241e0 2023-10-06T02:46:54.933 INFO:teuthology.orchestra.run.smithi033.stderr:error getting xattr cephfs_data/10000000001.00000000/parent: (2) No such file or directory 2023-10-06T02:46:54.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c871e241e0 msgr2=0x55c871ec36f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x55c871e241e0 0x55c871ec36f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f82a405eeb0 msgr2=0x7f82a4061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.933 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f82a405eeb0 0x7f82a4061360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f82b8001370 tx=0x7f82b8013000 comp rx=0 tx=0).stop 2023-10-06T02:46:54.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e269a0 msgr2=0x55c871ebb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 --2- 172.21.15.33:0/3084469957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c871e269a0 0x55c871ebb1c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82b0008a70 tx=0x7f82b000c040 comp rx=0 tx=0).stop 2023-10-06T02:46:54.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82be42f700 1 -- 172.21.15.33:0/3084469957 reap_dead start 2023-10-06T02:46:54.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 shutdown_connections 2023-10-06T02:46:54.934 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 >> 172.21.15.33:0/3084469957 conn(0x55c871dd70f0 msgr2=0x55c871dda410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 shutdown_connections 2023-10-06T02:46:54.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.929+0000 7f82c00f7e80 1 -- 172.21.15.33:0/3084469957 wait complete. 2023-10-06T02:46:54.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:54.937 ERROR:tasks.cephfs.filesystem:Command failed on smithi033 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent' 2023-10-06T02:46:54.937 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2023-10-06T02:46:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 Processor -- start 2023-10-06T02:46:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 -- start start 2023-10-06T02:46:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932630630 0x56193265ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 0x56193270add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932707740 0x56193270b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561932577840 con 0x561932658520 2023-10-06T02:46:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619325774e0 con 0x561932630630 2023-10-06T02:46:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e7ac2e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x56193256e270 con 0x561932707740 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 0x56193270add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 0x56193270add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59230/0 (socket says 172.21.15.33:59230) 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 -- 172.21.15.33:0/2820980731 learned_addr learned my addr 172.21.15.33:0/2820980731 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e55f9700 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932630630 0x56193265ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e5dfa700 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932707740 0x56193270b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 -- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932707740 msgr2=0x56193270b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932707740 0x56193270b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 -- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932630630 msgr2=0x56193265ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932630630 0x56193265ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 -- 172.21.15.33:0/2820980731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8005d50 con 0x561932658520 2023-10-06T02:46:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e5dfa700 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932707740 0x56193270b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 0x56193270add0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0001580 tx=0x7fb5e0006b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f2a10b488ce319 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5df7fe700 1 -- 172.21.15.33:0/2820980731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0006110 con 0x561932658520 2023-10-06T02:46:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5df7fe700 1 -- 172.21.15.33:0/2820980731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e0006270 con 0x561932658520 2023-10-06T02:46:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.965+0000 7fb5df7fe700 1 -- 172.21.15.33:0/2820980731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0013670 con 0x561932658520 2023-10-06T02:46:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/2820980731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 msgr2=0x56193270add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 0x56193270add0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0001580 tx=0x7fb5e0006b50 comp rx=0 tx=0).stop 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/2820980731 shutdown_connections 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932707740 0x56193270b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932658520 0x56193270add0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/2820980731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932630630 0x56193265ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/2820980731 >> 172.21.15.33:0/2820980731 conn(0x56193262ec90 msgr2=0x56193262b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/2820980731 shutdown_connections 2023-10-06T02:46:54.974 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/2820980731 wait complete. 2023-10-06T02:46:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 Processor -- start 2023-10-06T02:46:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.969+0000 7fb5e7ac2e80 1 -- start start 2023-10-06T02:46:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932630630 0x5619326fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932658520 0x5619326f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932707740 0x5619326fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193270aad0 con 0x561932707740 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619326f6f60 con 0x561932658520 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619326f7240 con 0x561932630630 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932630630 0x5619326fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932630630 0x5619326fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:35526/0 (socket says 172.21.15.33:35526) 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 -- 172.21.15.33:0/1679970450 learned_addr learned my addr 172.21.15.33:0/1679970450 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932658520 0x5619326f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e5dfa700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932707740 0x5619326fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 -- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932658520 msgr2=0x5619326f9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.978 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932658520 0x5619326f9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 -- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932707740 msgr2=0x5619326fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561932707740 0x5619326fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 -- 172.21.15.33:0/1679970450 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619325793e0 con 0x561932630630 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561932658520 0x5619326f9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e55f9700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932630630 0x5619326fc610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800cfd0 tx=0x7fb5d8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5ddffb700 1 -- 172.21.15.33:0/1679970450 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d801a050 con 0x561932630630 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5ddffb700 1 -- 172.21.15.33:0/1679970450 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d801b040 con 0x561932630630 2023-10-06T02:46:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561932586250 con 0x561932630630 2023-10-06T02:46:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5ddffb700 1 -- 172.21.15.33:0/1679970450 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8020460 con 0x561932630630 2023-10-06T02:46:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.973+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561932579770 con 0x561932630630 2023-10-06T02:46:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5ddffb700 1 -- 172.21.15.33:0/1679970450 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb5d8020850 con 0x561932630630 2023-10-06T02:46:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5ddffb700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb5cc05eeb0 0x7fb5cc061360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.981 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5ddffb700 1 -- 172.21.15.33:0/1679970450 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fb5d807ce80 con 0x561932630630 2023-10-06T02:46:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb5cc05eeb0 0x7fb5cc061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x5619326f59f0 0x5619326f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:54.982 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- osd_op(unknown.0.0:1 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5619326fcb50 con 0x5619326f59f0 2023-10-06T02:46:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5e5dfa700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x5619326f59f0 0x5619326f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5e4df8700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb5cc05eeb0 0x7fb5cc061360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0001580 tx=0x7fb5e0014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.977+0000 7fb5e5dfa700 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x5619326f59f0 0x5619326f8230 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e5dfa700 1 -- 172.21.15.33:0/1679970450 <== osd.3 v2:172.21.15.33:6817/33424 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=5b] v0'0 uv7 ondisk = 0) v8 ==== 154+0+5 (crc 0 0 0) 0x7fb5d0009b50 con 0x5619326f59f0 2023-10-06T02:46:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x5619326f59f0 msgr2=0x5619326f8230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x5619326f59f0 0x5619326f8230 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb5cc05eeb0 msgr2=0x7fb5cc061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb5cc05eeb0 0x7fb5cc061360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0001580 tx=0x7fb5e0014000 comp rx=0 tx=0).stop 2023-10-06T02:46:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932630630 msgr2=0x5619326fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 --2- 172.21.15.33:0/1679970450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x561932630630 0x5619326fc610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800cfd0 tx=0x7fb5d8014000 comp rx=0 tx=0).stop 2023-10-06T02:46:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e5dfa700 1 -- 172.21.15.33:0/1679970450 reap_dead start 2023-10-06T02:46:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 shutdown_connections 2023-10-06T02:46:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 >> 172.21.15.33:0/1679970450 conn(0x56193262ec90 msgr2=0x56193262c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 shutdown_connections 2023-10-06T02:46:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:54.981+0000 7fb5e7ac2e80 1 -- 172.21.15.33:0/1679970450 wait complete. 2023-10-06T02:46:54.988 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T02:46:54.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:46:55.007 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T02:46:55.029 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T02:46:55.029 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2023-10-06T02:46:55.029 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T02:46:55.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T02:46:55.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[44243]: fuse finished with error 0 and tester_r 0 2023-10-06T02:46:55.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:46:55.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:55.046 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T02:46:55.145 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:46:55.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:55.146 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T02:46:55.152 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T02:46:55.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:55.154 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:46:55.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:55.154 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T02:46:55.160 INFO:teuthology.orchestra.run.smithi033.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T02:46:55.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:55.161 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:46:55.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:46:55.162 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T02:46:55.168 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T02:46:55.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:46:55.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:55.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe362711700 1 Processor -- start 2023-10-06T02:46:55.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe362711700 1 -- start start 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe362711700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157280 0x7fe35c0aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe362711700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0cb6e0 con 0x7fe35c157650 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe362711700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0685d0 con 0x7fe35c0679c0 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe362711700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c6c10 con 0x7fe35c157280 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe360cae700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe35c157650 0x7fe35c157ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57292/0 (socket says 172.21.15.33:57292) 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe360cae700 1 -- 172.21.15.33:0/297896482 learned_addr learned my addr 172.21.15.33:0/297896482 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:55.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35bfff700 1 --2- 172.21.15.33:0/297896482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157280 0x7fe35c0aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240434948 0 0) 0x7fe35c0cb6e0 con 0x7fe35c157650 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe33c003570 con 0x7fe35c157650 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232047326 0 0) 0x7fe35c0685d0 con 0x7fe35c0679c0 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35c0cb6e0 con 0x7fe35c0679c0 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35bfff700 1 -- 172.21.15.33:0/297896482 >> v1:172.21.15.167:6789/0 conn(0x7fe35c0679c0 legacy=0x7fe35c067e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35bfff700 1 -- 172.21.15.33:0/297896482 >> v1:172.21.15.33:6789/0 conn(0x7fe35c157650 legacy=0x7fe35c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.281+0000 7fe35bfff700 1 -- 172.21.15.33:0/297896482 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0688b0 con 0x7fe35c157280 2023-10-06T02:46:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe35bfff700 1 --2- 172.21.15.33:0/297896482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157280 0x7fe35c0aadd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe344009910 tx=0x7fe3440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a604a78c5f17ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344011630 con 0x7fe35c157280 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe344011790 con 0x7fe35c157280 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe35affd700 1 -- 172.21.15.33:0/297896482 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344011b10 con 0x7fe35c157280 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 -- 172.21.15.33:0/297896482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157280 msgr2=0x7fe35c0aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 --2- 172.21.15.33:0/297896482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157280 0x7fe35c0aadd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe344009910 tx=0x7fe3440091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 -- 172.21.15.33:0/297896482 shutdown_connections 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 --2- 172.21.15.33:0/297896482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157280 0x7fe35c0aadd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 -- 172.21.15.33:0/297896482 >> 172.21.15.33:0/297896482 conn(0x7fe35c0c83f0 msgr2=0x7fe35c0c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 -- 172.21.15.33:0/297896482 shutdown_connections 2023-10-06T02:46:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 -- 172.21.15.33:0/297896482 wait complete. 2023-10-06T02:46:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 Processor -- start 2023-10-06T02:46:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.285+0000 7fe362711700 1 -- start start 2023-10-06T02:46:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0679c0 0x7fe35c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 0x7fe35c15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157650 0x7fe35c0c0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c6300 con 0x7fe35c157280 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c4960 con 0x7fe35c0679c0 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c43c0 con 0x7fe35c157650 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 0x7fe35c15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 0x7fe35c15bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59250/0 (socket says 172.21.15.33:59250) 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 -- 172.21.15.33:0/3760600940 learned_addr learned my addr 172.21.15.33:0/3760600940 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35bfff700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0679c0 0x7fe35c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe360cae700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157650 0x7fe35c0c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 -- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157650 msgr2=0x7fe35c0c0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157650 0x7fe35c0c0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 -- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0679c0 msgr2=0x7fe35c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0679c0 0x7fe35c0c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 -- 172.21.15.33:0/3760600940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe344009420 con 0x7fe35c157280 2023-10-06T02:46:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe35b7fe700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 0x7fe35c15bb10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe34c003260 tx=0x7fe34c002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c002020 con 0x7fe35c157280 2023-10-06T02:46:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0021c0 con 0x7fe35c157280 2023-10-06T02:46:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0a9650 con 0x7fe35c157280 2023-10-06T02:46:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c012880 con 0x7fe35c157280 2023-10-06T02:46:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c16c820 con 0x7fe35c157280 2023-10-06T02:46:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.289+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c16c680 con 0x7fe35c157280 2023-10-06T02:46:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.293+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe34c012bf0 con 0x7fe35c157280 2023-10-06T02:46:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.293+0000 7fe353fff700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe340060550 0x7fe340062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.293+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe34c07c8b0 con 0x7fe35c157280 2023-10-06T02:46:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.293+0000 7fe35bfff700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe340060550 0x7fe340062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.293+0000 7fe35bfff700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe340060550 0x7fe340062a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe344009860 tx=0x7fe344017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.301+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe35c16c680 con 0x7fe35c157280 2023-10-06T02:46:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.501+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35c16c680 con 0x7fe35c157280 2023-10-06T02:46:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.501+0000 7fe353fff700 1 -- 172.21.15.33:0/3760600940 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fe35c16c680 con 0x7fe35c157280 2023-10-06T02:46:55.506 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:55.506 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe340060550 msgr2=0x7fe340062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe340060550 0x7fe340062a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe344009860 tx=0x7fe344017000 comp rx=0 tx=0).stop 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 msgr2=0x7fe35c15bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 0x7fe35c15bb10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe34c003260 tx=0x7fe34c002720 comp rx=0 tx=0).stop 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 shutdown_connections 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe340060550 0x7fe340062a00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe35c157650 0x7fe35c0c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe35c157280 0x7fe35c15bb10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 --2- 172.21.15.33:0/3760600940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0679c0 0x7fe35c0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 >> 172.21.15.33:0/3760600940 conn(0x7fe35c0c83f0 msgr2=0x7fe35c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 shutdown_connections 2023-10-06T02:46:55.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.505+0000 7fe362711700 1 -- 172.21.15.33:0/3760600940 wait complete. 2023-10-06T02:46:55.510 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:55.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:55.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 Processor -- start 2023-10-06T02:46:55.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 -- start start 2023-10-06T02:46:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8620157280 0x7f86200aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8620157d60 0x7f8620067e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86200cb6e0 con 0x7f86200679c0 2023-10-06T02:46:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86200c6c10 con 0x7f8620157280 2023-10-06T02:46:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f8625194700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8620068380 con 0x7f8620157d60 2023-10-06T02:46:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8620157280 0x7f86200aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8620157280 0x7f86200aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59252/0 (socket says 172.21.15.33:59252) 2023-10-06T02:46:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 -- 172.21.15.33:0/2554476446 learned_addr learned my addr 172.21.15.33:0/2554476446 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861f59e700 1 --2- 172.21.15.33:0/2554476446 >> v2:172.21.15.167:3300/0 conn(0x7f8620157d60 0x7f8620067e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861dd9b700 1 -- 172.21.15.33:0/2554476446 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142709510 0 0) 0x7f86200cb6e0 con 0x7f86200679c0 2023-10-06T02:46:55.634 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 -- 172.21.15.33:0/2554476446 >> v2:172.21.15.167:3300/0 conn(0x7f8620157d60 msgr2=0x7f8620067e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 --2- 172.21.15.33:0/2554476446 >> v2:172.21.15.167:3300/0 conn(0x7f8620157d60 0x7f8620067e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 -- 172.21.15.33:0/2554476446 >> v1:172.21.15.167:6789/0 conn(0x7f86200679c0 legacy=0x7f8620157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 -- 172.21.15.33:0/2554476446 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200685d0 con 0x7f8620157280 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.625+0000 7f861ed9d700 1 --2- 172.21.15.33:0/2554476446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8620157280 0x7f86200aadd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f86200c77d0 tx=0x7f8610009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d5b33d5280b19e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f861dd9b700 1 -- 172.21.15.33:0/2554476446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610009e70 con 0x7f8620157280 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f861dd9b700 1 -- 172.21.15.33:0/2554476446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861000b860 con 0x7f8620157280 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f861dd9b700 1 -- 172.21.15.33:0/2554476446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000bc00 con 0x7f8620157280 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 -- 172.21.15.33:0/2554476446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8620157280 msgr2=0x7f86200aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 --2- 172.21.15.33:0/2554476446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8620157280 0x7f86200aadd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f86200c77d0 tx=0x7f8610009250 comp rx=0 tx=0).stop 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 -- 172.21.15.33:0/2554476446 shutdown_connections 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 --2- 172.21.15.33:0/2554476446 >> v2:172.21.15.167:3300/0 conn(0x7f8620157d60 0x7f8620067e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.635 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 --2- 172.21.15.33:0/2554476446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8620157280 0x7f86200aadd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 -- 172.21.15.33:0/2554476446 >> 172.21.15.33:0/2554476446 conn(0x7f86200c83f0 msgr2=0x7f86200c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:55.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 -- 172.21.15.33:0/2554476446 shutdown_connections 2023-10-06T02:46:55.636 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.629+0000 7f8625194700 1 -- 172.21.15.33:0/2554476446 wait complete. 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 Processor -- start 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 -- start start 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 0x7f86200a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8620157280 0x7f86200a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8620157d60 0x7f86200c0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200c27e0 con 0x7f86200679c0 2023-10-06T02:46:55.637 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200c1fd0 con 0x7f8620157d60 2023-10-06T02:46:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200aa440 con 0x7f8620157280 2023-10-06T02:46:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 0x7f86200a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 0x7f86200a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59256/0 (socket says 172.21.15.33:59256) 2023-10-06T02:46:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 -- 172.21.15.33:0/3619312172 learned_addr learned my addr 172.21.15.33:0/3619312172 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861e59c700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8620157280 0x7f86200a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861f59e700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8620157d60 0x7f86200c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 -- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8620157280 msgr2=0x7f86200a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8620157280 0x7f86200a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 -- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8620157d60 msgr2=0x7f86200c0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8620157d60 0x7f86200c0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 -- 172.21.15.33:0/3619312172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86100094f0 con 0x7f86200679c0 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861e59c700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8620157280 0x7f86200a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861f59e700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8620157d60 0x7f86200c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f861ed9d700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 0x7f86200a9960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f86100094c0 tx=0x7f8610017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.640 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000b9d0 con 0x7f86200679c0 2023-10-06T02:46:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861000bb50 con 0x7f86200679c0 2023-10-06T02:46:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200a9460 con 0x7f86200679c0 2023-10-06T02:46:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610025670 con 0x7f86200679c0 2023-10-06T02:46:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.633+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86200c2500 con 0x7f86200679c0 2023-10-06T02:46:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.637+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86200710b0 con 0x7f86200679c0 2023-10-06T02:46:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.637+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8610023070 con 0x7f86200679c0 2023-10-06T02:46:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.637+0000 7f8616ffd700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8600060550 0x7f8600062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.637+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f861007dff0 con 0x7f86200679c0 2023-10-06T02:46:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.637+0000 7f861e59c700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8600060550 0x7f8600062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.637+0000 7f861e59c700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8600060550 0x7f8600062a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8608007fd0 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.645+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f861004e340 con 0x7f86200679c0 2023-10-06T02:46:55.815 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.809+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86200ac090 con 0x7f86200679c0 2023-10-06T02:46:55.817 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8616ffd700 1 -- 172.21.15.33:0/3619312172 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17071 (secure 0 0 0) 0x7f8610051960 con 0x7f86200679c0 2023-10-06T02:46:55.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":44,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:29.309782+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:46:25.764531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T02:46:26.918595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:55.819 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8600060550 msgr2=0x7f8600062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8600060550 0x7f8600062a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8608007fd0 tx=0x7f8608009040 comp rx=0 tx=0).stop 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 msgr2=0x7f86200a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 0x7f86200a9960 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f86100094c0 tx=0x7f8610017000 comp rx=0 tx=0).stop 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 shutdown_connections 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8600060550 0x7f8600062a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8620157280 0x7f86200a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8620157d60 0x7f86200c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 --2- 172.21.15.33:0/3619312172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86200679c0 0x7f86200a9960 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.813+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 >> 172.21.15.33:0/3619312172 conn(0x7f86200c83f0 msgr2=0x7f862006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.817+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 shutdown_connections 2023-10-06T02:46:55.820 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.817+0000 7f8625194700 1 -- 172.21.15.33:0/3619312172 wait complete. 2023-10-06T02:46:55.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T02:45:10.622496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T02:45:16.980990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T02:46:25.764531+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T02:46:26.918595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T02:46:55.833 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T02:46:55.948 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 Processor -- start 2023-10-06T02:46:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 -- start start 2023-10-06T02:46:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55d0154e20 0x7f55d00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d00c7590 con 0x7f55d0066fb0 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d006b230 con 0x7f55d0154e20 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d549a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d006b510 con 0x7f55d01551f0 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55cffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55d0154e20 0x7f55d00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55cffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55d0154e20 0x7f55d00caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59260/0 (socket says 172.21.15.33:59260) 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55cffff700 1 -- 172.21.15.33:0/4142447519 learned_addr learned my addr 172.21.15.33:0/4142447519 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.945+0000 7f55d4c99700 1 --2- 172.21.15.33:0/4142447519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cffff700 1 -- 172.21.15.33:0/4142447519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 msgr2=0x7f55d00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cffff700 1 --2- 172.21.15.33:0/4142447519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cffff700 1 -- 172.21.15.33:0/4142447519 >> v1:172.21.15.167:6789/0 conn(0x7f55d0066fb0 legacy=0x7f55d006a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cffff700 1 -- 172.21.15.33:0/4142447519 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d006b7f0 con 0x7f55d0154e20 2023-10-06T02:46:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cffff700 1 --2- 172.21.15.33:0/4142447519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0154e20 0x7f55d00caa70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f55c000ab50 tx=0x7f55c000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539a1b80069fcf5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cf7fe700 1 -- 172.21.15.33:0/4142447519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c000ca60 con 0x7f55d0154e20 2023-10-06T02:46:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cf7fe700 1 -- 172.21.15.33:0/4142447519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000cbc0 con 0x7f55d0154e20 2023-10-06T02:46:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55cf7fe700 1 -- 172.21.15.33:0/4142447519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0018400 con 0x7f55d0154e20 2023-10-06T02:46:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 -- 172.21.15.33:0/4142447519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0154e20 msgr2=0x7f55d00caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 --2- 172.21.15.33:0/4142447519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0154e20 0x7f55d00caa70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f55c000ab50 tx=0x7f55c000a3f0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 -- 172.21.15.33:0/4142447519 shutdown_connections 2023-10-06T02:46:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 --2- 172.21.15.33:0/4142447519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 --2- 172.21.15.33:0/4142447519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0154e20 0x7f55d00caa70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 -- 172.21.15.33:0/4142447519 >> 172.21.15.33:0/4142447519 conn(0x7f55d00c8ad0 msgr2=0x7f55d00c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 -- 172.21.15.33:0/4142447519 shutdown_connections 2023-10-06T02:46:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.949+0000 7f55d549a700 1 -- 172.21.15.33:0/4142447519 wait complete. 2023-10-06T02:46:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 Processor -- start 2023-10-06T02:46:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 -- start start 2023-10-06T02:46:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 0x7f55d0167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d0154e20 0x7f55d01531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d0149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d00a4bb0 con 0x7f55d0066fb0 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d0167770 con 0x7f55d0154e20 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d00a6190 con 0x7f55d01551f0 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 0x7f55d0167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 0x7f55d0167230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59268/0 (socket says 172.21.15.33:59268) 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 -- 172.21.15.33:0/2631029223 learned_addr learned my addr 172.21.15.33:0/2631029223 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d4c99700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d0149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.960 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55c77fe700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d0154e20 0x7f55d01531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 -- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 msgr2=0x7f55d0149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d0149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 -- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d0154e20 msgr2=0x7f55d01531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d0154e20 0x7f55d01531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 -- 172.21.15.33:0/2631029223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c000a690 con 0x7f55d0066fb0 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55c77fe700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d0154e20 0x7f55d01531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55cffff700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 0x7f55d0167230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f55c000a660 tx=0x7f55c001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0010600 con 0x7f55d0066fb0 2023-10-06T02:46:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c0010760 con 0x7f55d0066fb0 2023-10-06T02:46:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d00a4560 con 0x7f55d0066fb0 2023-10-06T02:46:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.953+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0010b20 con 0x7f55d0066fb0 2023-10-06T02:46:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.957+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d00a6470 con 0x7f55d0066fb0 2023-10-06T02:46:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.957+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d0071040 con 0x7f55d0066fb0 2023-10-06T02:46:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.957+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f55c002a3b0 con 0x7f55d0066fb0 2023-10-06T02:46:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.957+0000 7f55ccff9700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55b0060550 0x7f55b0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.957+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f55c007e170 con 0x7f55d0066fb0 2023-10-06T02:46:55.963 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.957+0000 7f55c77fe700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55b0060550 0x7f55b0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.961+0000 7f55c77fe700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55b0060550 0x7f55b0062a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55b8007fd0 tx=0x7f55b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:55.965+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f55c004e420 con 0x7f55d0066fb0 2023-10-06T02:46:56.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.129+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55d00c7120 con 0x7f55d0066fb0 2023-10-06T02:46:56.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55ccff9700 1 -- 172.21.15.33:0/2631029223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55c0051a40 con 0x7f55d0066fb0 2023-10-06T02:46:56.136 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2023-10-06T02:46:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55b0060550 msgr2=0x7f55b0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55b0060550 0x7f55b0062a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55b8007fd0 tx=0x7f55b8009040 comp rx=0 tx=0).stop 2023-10-06T02:46:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 msgr2=0x7f55d0167230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 0x7f55d0167230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f55c000a660 tx=0x7f55c001a000 comp rx=0 tx=0).stop 2023-10-06T02:46:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 shutdown_connections 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f55b0060550 0x7f55b0062a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55d01551f0 0x7f55d0149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55d0154e20 0x7f55d01531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 --2- 172.21.15.33:0/2631029223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55d0066fb0 0x7f55d0167230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 >> 172.21.15.33:0/2631029223 conn(0x7f55d00c8ad0 msgr2=0x7f55d00c13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 shutdown_connections 2023-10-06T02:46:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.133+0000 7f55d549a700 1 -- 172.21.15.33:0/2631029223 wait complete. 2023-10-06T02:46:56.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T02:46:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 Processor -- start 2023-10-06T02:46:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 -- start start 2023-10-06T02:46:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2fa8159030 0x7f2fa80caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa80cb020 0x7f2fa80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa80cd7f0 con 0x7f2fa8158c60 2023-10-06T02:46:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa80cddb0 con 0x7f2fa8159030 2023-10-06T02:46:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8159480 con 0x7f2fa80cb020 2023-10-06T02:46:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2fa8158c60 0x7f2fa815cb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57300/0 (socket says 172.21.15.33:57300) 2023-10-06T02:46:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa659c700 1 -- 172.21.15.33:0/1537191125 learned_addr learned my addr 172.21.15.33:0/1537191125 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/1537191125 >> v2:172.21.15.167:3300/0 conn(0x7f2fa8159030 0x7f2fa80caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.257+0000 7f2fa6d9d700 1 --2- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa80cb020 0x7f2fa80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa559a700 1 -- 172.21.15.33:0/1537191125 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943384527 0 0) 0x7f2fa80cd7f0 con 0x7f2fa8158c60 2023-10-06T02:46:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa559a700 1 -- 172.21.15.33:0/1537191125 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f90003570 con 0x7f2fa8158c60 2023-10-06T02:46:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa5d9b700 1 -- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa80cb020 msgr2=0x7f2fa80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa80cb020 0x7f2fa80ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa5d9b700 1 -- 172.21.15.33:0/1537191125 >> v1:172.21.15.33:6789/0 conn(0x7f2fa8158c60 legacy=0x7f2fa815cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa5d9b700 1 -- 172.21.15.33:0/1537191125 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa815d630 con 0x7f2fa8159030 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8159030 0x7f2fa80caae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f980098e0 tx=0x7f2f980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1fa6d2838425d06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa559a700 1 -- 172.21.15.33:0/1537191125 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98011630 con 0x7f2fa8159030 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa559a700 1 -- 172.21.15.33:0/1537191125 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98011790 con 0x7f2fa8159030 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa559a700 1 -- 172.21.15.33:0/1537191125 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98011b10 con 0x7f2fa8159030 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 -- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8159030 msgr2=0x7f2fa80caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 --2- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8159030 0x7f2fa80caae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f980098e0 tx=0x7f2f980091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 -- 172.21.15.33:0/1537191125 shutdown_connections 2023-10-06T02:46:56.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 --2- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa80cb020 0x7f2fa80ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 --2- 172.21.15.33:0/1537191125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8159030 0x7f2fa80caae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 -- 172.21.15.33:0/1537191125 >> 172.21.15.33:0/1537191125 conn(0x7f2fa8067a80 msgr2=0x7f2fa80cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 -- 172.21.15.33:0/1537191125 shutdown_connections 2023-10-06T02:46:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.261+0000 7f2fa759e700 1 -- 172.21.15.33:0/1537191125 wait complete. 2023-10-06T02:46:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 Processor -- start 2023-10-06T02:46:56.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 -- start start 2023-10-06T02:46:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 0x7f2fa814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8158c60 0x7f2fa80b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa8159030 0x7f2fa80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa814f000 con 0x7f2fa80cb020 2023-10-06T02:46:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80b0fd0 con 0x7f2fa8158c60 2023-10-06T02:46:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80b03c0 con 0x7f2fa8159030 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 0x7f2fa814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 0x7f2fa814eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59270/0 (socket says 172.21.15.33:59270) 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 -- 172.21.15.33:0/983765292 learned_addr learned my addr 172.21.15.33:0/983765292 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8158c60 0x7f2fa80b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa6d9d700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa8159030 0x7f2fa80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 -- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa8159030 msgr2=0x7f2fa80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa8159030 0x7f2fa80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 -- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8158c60 msgr2=0x7f2fa80b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8158c60 0x7f2fa80b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 -- 172.21.15.33:0/983765292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98009420 con 0x7f2fa80cb020 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8158c60 0x7f2fa80b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa659c700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 0x7f2fa814eac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0011e0 tx=0x7f2f9c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c005b30 con 0x7f2fa80cb020 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c005c90 con 0x7f2fa80cb020 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa80b06a0 con 0x7f2fa80cb020 2023-10-06T02:46:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c007300 con 0x7f2fa80cb020 2023-10-06T02:46:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.265+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa80b0c60 con 0x7f2fa80cb020 2023-10-06T02:46:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.269+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa8071040 con 0x7f2fa80cb020 2023-10-06T02:46:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.269+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2f9c007670 con 0x7f2fa80cb020 2023-10-06T02:46:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.269+0000 7f2f967fc700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2f88060550 0x7f2f88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.269+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2f9c07c5c0 con 0x7f2fa80cb020 2023-10-06T02:46:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.269+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2f88060550 0x7f2f88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.269+0000 7f2fa5d9b700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2f88060550 0x7f2f88062a00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f980091e0 tx=0x7f2f98017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.281+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f9c04c990 con 0x7f2fa80cb020 2023-10-06T02:46:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.445+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2fa815d910 con 0x7f2fa80cb020 2023-10-06T02:46:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.445+0000 7f2f967fc700 1 -- 172.21.15.33:0/983765292 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2f9c04ffb0 con 0x7f2fa80cb020 2023-10-06T02:46:56.451 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2f88060550 msgr2=0x7f2f88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2f88060550 0x7f2f88062a00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f980091e0 tx=0x7f2f98017000 comp rx=0 tx=0).stop 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 msgr2=0x7f2fa814eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 0x7f2fa814eac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0011e0 tx=0x7f2f9c007df0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 shutdown_connections 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2f88060550 0x7f2f88062a00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2fa8159030 0x7f2fa80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fa8158c60 0x7f2fa80b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.454 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 --2- 172.21.15.33:0/983765292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa80cb020 0x7f2fa814eac0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 >> 172.21.15.33:0/983765292 conn(0x7f2fa8067a80 msgr2=0x7f2fa80ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 shutdown_connections 2023-10-06T02:46:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.449+0000 7f2fa759e700 1 -- 172.21.15.33:0/983765292 wait complete. 2023-10-06T02:46:56.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T02:46:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.573+0000 7f16fcf50700 1 Processor -- start 2023-10-06T02:46:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.573+0000 7f16fcf50700 1 -- start start 2023-10-06T02:46:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f8158c60 0x7f16f8159030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16f815cb70 0x7f16f815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f80cd7f0 con 0x7f16f80caae0 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f80cddb0 con 0x7f16f815cb70 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f815d630 con 0x7f16f8158c60 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16f815cb70 0x7f16f815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f16f815cb70 0x7f16f815cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59284/0 (socket says 172.21.15.33:59284) 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 -- 172.21.15.33:0/3343999407 learned_addr learned my addr 172.21.15.33:0/3343999407 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f77fe700 1 --2- 172.21.15.33:0/3343999407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f8158c60 0x7f16f8159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 -- 172.21.15.33:0/3343999407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f8158c60 msgr2=0x7f16f8159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 --2- 172.21.15.33:0/3343999407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f8158c60 0x7f16f8159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 -- 172.21.15.33:0/3343999407 >> v1:172.21.15.167:6789/0 conn(0x7f16f80caae0 legacy=0x7f16f80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 -- 172.21.15.33:0/3343999407 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f815d910 con 0x7f16f815cb70 2023-10-06T02:46:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f6ffd700 1 --2- 172.21.15.33:0/3343999407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f815cb70 0x7f16f815cfc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f16e80099b0 tx=0x7f16e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa76504fd15717f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f67fc700 1 -- 172.21.15.33:0/3343999407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8009ea0 con 0x7f16f815cb70 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f67fc700 1 -- 172.21.15.33:0/3343999407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e800b860 con 0x7f16f815cb70 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16f67fc700 1 -- 172.21.15.33:0/3343999407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800bc00 con 0x7f16f815cb70 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- 172.21.15.33:0/3343999407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f815cb70 msgr2=0x7f16f815cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 --2- 172.21.15.33:0/3343999407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f815cb70 0x7f16f815cfc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f16e80099b0 tx=0x7f16e8009250 comp rx=0 tx=0).stop 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- 172.21.15.33:0/3343999407 shutdown_connections 2023-10-06T02:46:56.588 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 --2- 172.21.15.33:0/3343999407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f815cb70 0x7f16f815cfc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 --2- 172.21.15.33:0/3343999407 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f8158c60 0x7f16f8159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- 172.21.15.33:0/3343999407 >> 172.21.15.33:0/3343999407 conn(0x7f16f8067a80 msgr2=0x7f16f80cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- 172.21.15.33:0/3343999407 shutdown_connections 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- 172.21.15.33:0/3343999407 wait complete. 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 Processor -- start 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.577+0000 7f16fcf50700 1 -- start start 2023-10-06T02:46:56.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f80caae0 0x7f16f814caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 0x7f16f814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f815cb70 0x7f16f814e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8151790 con 0x7f16f8158c60 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8151a70 con 0x7f16f815cb70 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f8151d50 con 0x7f16f80caae0 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 0x7f16f814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 0x7f16f814e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59288/0 (socket says 172.21.15.33:59288) 2023-10-06T02:46:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 -- 172.21.15.33:0/2318034837 learned_addr learned my addr 172.21.15.33:0/2318034837 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f77fe700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f80caae0 0x7f16f814caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f7fff700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f815cb70 0x7f16f814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 -- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f80caae0 msgr2=0x7f16f814caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f80caae0 0x7f16f814caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 -- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f815cb70 msgr2=0x7f16f814e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f815cb70 0x7f16f814e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 -- 172.21.15.33:0/2318034837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80094f0 con 0x7f16f8158c60 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16f6ffd700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 0x7f16f814e230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16f805e870 tx=0x7f16e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800bbe0 con 0x7f16f8158c60 2023-10-06T02:46:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e8011420 con 0x7f16f8158c60 2023-10-06T02:46:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f8152030 con 0x7f16f8158c60 2023-10-06T02:46:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80117c0 con 0x7f16f8158c60 2023-10-06T02:46:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f8150a40 con 0x7f16f8158c60 2023-10-06T02:46:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.581+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f8071040 con 0x7f16f8158c60 2023-10-06T02:46:56.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.585+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f16e8011bf0 con 0x7f16f8158c60 2023-10-06T02:46:56.593 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.585+0000 7f16df7fe700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f16e4060550 0x7f16e4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.585+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f16e807e230 con 0x7f16f8158c60 2023-10-06T02:46:56.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.585+0000 7f16f77fe700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f16e4060550 0x7f16e4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.594 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.585+0000 7f16f77fe700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f16e4060550 0x7f16e4062a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009db0 tx=0x7f16ec0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.593+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f16e804e600 con 0x7f16f8158c60 2023-10-06T02:46:56.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.761+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16f80cda40 con 0x7f16f8158c60 2023-10-06T02:46:56.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.761+0000 7f16df7fe700 1 -- 172.21.15.33:0/2318034837 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16e8051ba0 con 0x7f16f8158c60 2023-10-06T02:46:56.766 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2023-10-06T02:46:56.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f16e4060550 msgr2=0x7f16e4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f16e4060550 0x7f16e4062a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009db0 tx=0x7f16ec0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 msgr2=0x7f16f814e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 0x7f16f814e230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16f805e870 tx=0x7f16e8017000 comp rx=0 tx=0).stop 2023-10-06T02:46:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 shutdown_connections 2023-10-06T02:46:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f815cb70 0x7f16f814e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.771 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16f8158c60 0x7f16f814e230 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f16e4060550 0x7f16e4062a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 --2- 172.21.15.33:0/2318034837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f16f80caae0 0x7f16f814caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 >> 172.21.15.33:0/2318034837 conn(0x7f16f8067a80 msgr2=0x7f16f80c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 shutdown_connections 2023-10-06T02:46:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.765+0000 7f16fcf50700 1 -- 172.21.15.33:0/2318034837 wait complete. 2023-10-06T02:46:56.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T02:46:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.889+0000 7f6e84a22700 1 Processor -- start 2023-10-06T02:46:56.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.889+0000 7f6e84a22700 1 -- start start 2023-10-06T02:46:56.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e800b1300 0x7f6e80066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800ab230 con 0x7f6e8006a4e0 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800ab510 con 0x7f6e80069d90 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8006aa80 con 0x7f6e800b1300 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6e8006a4e0 0x7f6e800ad710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57314/0 (socket says 172.21.15.33:57314) 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7ffff700 1 -- 172.21.15.33:0/2005078529 learned_addr learned my addr 172.21.15.33:0/2005078529 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e77fff700 1 --2- 172.21.15.33:0/2005078529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e800b1300 0x7f6e80066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7effd700 1 -- 172.21.15.33:0/2005078529 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944001068 0 0) 0x7f6e800ab230 con 0x7f6e8006a4e0 2023-10-06T02:46:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7effd700 1 -- 172.21.15.33:0/2005078529 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e6c003570 con 0x7f6e8006a4e0 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e77fff700 1 -- 172.21.15.33:0/2005078529 >> v1:172.21.15.167:6789/0 conn(0x7f6e80069d90 legacy=0x7f6e800ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7effd700 1 -- 172.21.15.33:0/2005078529 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022589712 0 0) 0x7f6e800ab510 con 0x7f6e80069d90 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e77fff700 1 -- 172.21.15.33:0/2005078529 >> v1:172.21.15.33:6789/0 conn(0x7f6e8006a4e0 legacy=0x7f6e800ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e77fff700 1 -- 172.21.15.33:0/2005078529 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800c8d40 con 0x7f6e800b1300 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e77fff700 1 --2- 172.21.15.33:0/2005078529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e800b1300 0x7f6e80066fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009910 tx=0x7f6e680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc3c4cf944f6d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7effd700 1 -- 172.21.15.33:0/2005078529 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800b680 con 0x7f6e800b1300 2023-10-06T02:46:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7effd700 1 -- 172.21.15.33:0/2005078529 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800b7e0 con 0x7f6e800b1300 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e7effd700 1 -- 172.21.15.33:0/2005078529 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800bb60 con 0x7f6e800b1300 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- 172.21.15.33:0/2005078529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e800b1300 msgr2=0x7f6e80066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 --2- 172.21.15.33:0/2005078529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e800b1300 0x7f6e80066fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009910 tx=0x7f6e680091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- 172.21.15.33:0/2005078529 shutdown_connections 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 --2- 172.21.15.33:0/2005078529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e800b1300 0x7f6e80066fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- 172.21.15.33:0/2005078529 >> 172.21.15.33:0/2005078529 conn(0x7f6e800cc2c0 msgr2=0x7f6e800c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.893+0000 7f6e84a22700 1 -- 172.21.15.33:0/2005078529 shutdown_connections 2023-10-06T02:46:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 -- 172.21.15.33:0/2005078529 wait complete. 2023-10-06T02:46:56.906 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 Processor -- start 2023-10-06T02:46:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 -- start start 2023-10-06T02:46:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e80069d90 0x7f6e800c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8006a4e0 0x7f6e80157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 0x7f6e800c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800a4cc0 con 0x7f6e800b1300 2023-10-06T02:46:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80167770 con 0x7f6e80069d90 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e84a22700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80167a10 con 0x7f6e8006a4e0 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.897+0000 7f6e7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 0x7f6e800c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 0x7f6e800c28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59298/0 (socket says 172.21.15.33:59298) 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 -- 172.21.15.33:0/3049808239 learned_addr learned my addr 172.21.15.33:0/3049808239 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e77fff700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8006a4e0 0x7f6e80157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7f7fe700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e80069d90 0x7f6e800c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 -- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8006a4e0 msgr2=0x7f6e80157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8006a4e0 0x7f6e80157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 -- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e80069d90 msgr2=0x7f6e800c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e80069d90 0x7f6e800c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 -- 172.21.15.33:0/3049808239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009420 con 0x7f6e800b1300 2023-10-06T02:46:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e77fff700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8006a4e0 0x7f6e80157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e7ffff700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 0x7f6e800c28a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e78001460 tx=0x7f6e78001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7800a870 con 0x7f6e800b1300 2023-10-06T02:46:56.909 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7800a9d0 con 0x7f6e800b1300 2023-10-06T02:46:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e800a4670 con 0x7f6e800b1300 2023-10-06T02:46:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7800ad50 con 0x7f6e800b1300 2023-10-06T02:46:56.910 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.901+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e80157690 con 0x7f6e800b1300 2023-10-06T02:46:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.905+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6e78007300 con 0x7f6e800b1300 2023-10-06T02:46:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.905+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e800a44d0 con 0x7f6e800b1300 2023-10-06T02:46:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.905+0000 7f6e777fe700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6e60060550 0x7f6e60062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.905+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f6e7807cd80 con 0x7f6e800b1300 2023-10-06T02:46:56.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.905+0000 7f6e7f7fe700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6e60060550 0x7f6e60062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:56.913 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.905+0000 7f6e7f7fe700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6e60060550 0x7f6e60062a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6e700039f0 tx=0x7f6e70007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:56.920 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:56.917+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6e800a44d0 con 0x7f6e800b1300 2023-10-06T02:46:57.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e800a44d0 con 0x7f6e800b1300 2023-10-06T02:46:57.088 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e777fe700 1 -- 172.21.15.33:0/3049808239 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e800a44d0 con 0x7f6e800b1300 2023-10-06T02:46:57.089 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2023-10-06T02:46:57.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6e60060550 msgr2=0x7f6e60062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6e60060550 0x7f6e60062a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6e700039f0 tx=0x7f6e70007ea0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 msgr2=0x7f6e800c28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 0x7f6e800c28a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e78001460 tx=0x7f6e78001a40 comp rx=0 tx=0).stop 2023-10-06T02:46:57.091 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 shutdown_connections 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6e60060550 0x7f6e60062a00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8006a4e0 0x7f6e80157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e800b1300 0x7f6e800c28a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 --2- 172.21.15.33:0/3049808239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e80069d90 0x7f6e800c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 >> 172.21.15.33:0/3049808239 conn(0x7f6e800cc2c0 msgr2=0x7f6e800aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 shutdown_connections 2023-10-06T02:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.085+0000 7f6e84a22700 1 -- 172.21.15.33:0/3049808239 wait complete. 2023-10-06T02:46:57.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 Processor -- start 2023-10-06T02:46:57.208 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 -- start start 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe214159fa0 0x7fe2140cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe2140ccc40 0x7fe2140ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2140cb5b0 con 0x7fe214158e70 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21415a410 con 0x7fe214159fa0 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe21a508700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe214159240 con 0x7fe2140ccc40 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe214159fa0 0x7fe2140cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe214159fa0 0x7fe2140cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59308/0 (socket says 172.21.15.33:59308) 2023-10-06T02:46:57.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 -- 172.21.15.33:0/974042126 learned_addr learned my addr 172.21.15.33:0/974042126 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:57.210 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe218aa5700 1 --2- 172.21.15.33:0/974042126 >> v2:172.21.15.167:3300/0 conn(0x7fe2140ccc40 0x7fe2140ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe2137fe700 1 -- 172.21.15.33:0/974042126 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764170240 0 0) 0x7fe2140cb5b0 con 0x7fe214158e70 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe2137fe700 1 -- 172.21.15.33:0/974042126 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe200003570 con 0x7fe214158e70 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 -- 172.21.15.33:0/974042126 >> v2:172.21.15.167:3300/0 conn(0x7fe2140ccc40 msgr2=0x7fe2140ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 --2- 172.21.15.33:0/974042126 >> v2:172.21.15.167:3300/0 conn(0x7fe2140ccc40 0x7fe2140ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 -- 172.21.15.33:0/974042126 >> v1:172.21.15.167:6789/0 conn(0x7fe214158e70 legacy=0x7fe2140cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 -- 172.21.15.33:0/974042126 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140cda80 con 0x7fe214159fa0 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe20b7fe700 1 --2- 172.21.15.33:0/974042126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe214159fa0 0x7fe2140cc700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe2140890c0 tx=0x7fe1fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2d40fcb957e4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe2137fe700 1 -- 172.21.15.33:0/974042126 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00b640 con 0x7fe214159fa0 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe2137fe700 1 -- 172.21.15.33:0/974042126 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00b7a0 con 0x7fe214159fa0 2023-10-06T02:46:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.205+0000 7fe2137fe700 1 -- 172.21.15.33:0/974042126 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00bb40 con 0x7fe214159fa0 2023-10-06T02:46:57.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 -- 172.21.15.33:0/974042126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe214159fa0 msgr2=0x7fe2140cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 --2- 172.21.15.33:0/974042126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe214159fa0 0x7fe2140cc700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe2140890c0 tx=0x7fe1fc009250 comp rx=0 tx=0).stop 2023-10-06T02:46:57.212 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 -- 172.21.15.33:0/974042126 shutdown_connections 2023-10-06T02:46:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 --2- 172.21.15.33:0/974042126 >> v2:172.21.15.167:3300/0 conn(0x7fe2140ccc40 0x7fe2140ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 --2- 172.21.15.33:0/974042126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe214159fa0 0x7fe2140cc700 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 -- 172.21.15.33:0/974042126 >> 172.21.15.33:0/974042126 conn(0x7fe214067a80 msgr2=0x7fe214068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 -- 172.21.15.33:0/974042126 shutdown_connections 2023-10-06T02:46:57.213 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 -- 172.21.15.33:0/974042126 wait complete. 2023-10-06T02:46:57.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 Processor -- start 2023-10-06T02:46:57.214 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.209+0000 7fe21a508700 1 -- start start 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 0x7fe2140b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe214158e70 0x7fe2140b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe214159fa0 0x7fe2140b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415bba0 con 0x7fe2140ccc40 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141541b0 con 0x7fe214159fa0 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140b59a0 con 0x7fe214158e70 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 0x7fe2140b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 0x7fe2140b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59316/0 (socket says 172.21.15.33:59316) 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 -- 172.21.15.33:0/4062420449 learned_addr learned my addr 172.21.15.33:0/4062420449 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe20b7fe700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe214158e70 0x7fe2140b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe218aa5700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe214159fa0 0x7fe2140b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 -- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe214158e70 msgr2=0x7fe2140b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe214158e70 0x7fe2140b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 -- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe214159fa0 msgr2=0x7fe2140b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe214159fa0 0x7fe2140b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 -- 172.21.15.33:0/4062420449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0094f0 con 0x7fe2140ccc40 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe213fff700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 0x7fe2140b4640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe20400c5b0 tx=0x7fe204007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204010510 con 0x7fe2140ccc40 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204010690 con 0x7fe2140ccc40 2023-10-06T02:46:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe214169900 con 0x7fe2140ccc40 2023-10-06T02:46:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204010a50 con 0x7fe2140ccc40 2023-10-06T02:46:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe214169ec0 con 0x7fe2140ccc40 2023-10-06T02:46:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.213+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214088b10 con 0x7fe2140ccc40 2023-10-06T02:46:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.217+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe20401b400 con 0x7fe2140ccc40 2023-10-06T02:46:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.217+0000 7fe210ff9700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1f4060580 0x7fe1f4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.217+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fe20407c730 con 0x7fe2140ccc40 2023-10-06T02:46:57.222 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.217+0000 7fe20b7fe700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1f4060580 0x7fe1f4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.217+0000 7fe20b7fe700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1f4060580 0x7fe1f4062a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe2140890c0 tx=0x7fe1fc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.225+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe214088b10 con 0x7fe2140ccc40 2023-10-06T02:46:57.427 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.421+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe214088b10 con 0x7fe2140ccc40 2023-10-06T02:46:57.428 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe210ff9700 1 -- 172.21.15.33:0/4062420449 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fe214088b10 con 0x7fe2140ccc40 2023-10-06T02:46:57.429 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1f4060580 msgr2=0x7fe1f4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1f4060580 0x7fe1f4062a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe2140890c0 tx=0x7fe1fc017000 comp rx=0 tx=0).stop 2023-10-06T02:46:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 msgr2=0x7fe2140b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 0x7fe2140b4640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe20400c5b0 tx=0x7fe204007e10 comp rx=0 tx=0).stop 2023-10-06T02:46:57.431 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 shutdown_connections 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fe1f4060580 0x7fe1f4062a30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe214158e70 0x7fe2140b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe214159fa0 0x7fe2140b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 --2- 172.21.15.33:0/4062420449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2140ccc40 0x7fe2140b4640 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 >> 172.21.15.33:0/4062420449 conn(0x7fe214067a80 msgr2=0x7fe2140ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 shutdown_connections 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.425+0000 7fe21a508700 1 -- 172.21.15.33:0/4062420449 wait complete. 2023-10-06T02:46:57.432 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:57.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:57.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.537+0000 7feee84aa700 1 Processor -- start 2023-10-06T02:46:57.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.537+0000 7feee84aa700 1 -- start start 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee00b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feee00b0530 0x7feee00b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feee0158fc0 0x7feee015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee0067c10 con 0x7feee0158fc0 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee00b2610 con 0x7feee00b0530 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee015d830 con 0x7feee0158bf0 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feee0158fc0 0x7feee015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7feee0158fc0 0x7feee015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59318/0 (socket says 172.21.15.33:59318) 2023-10-06T02:46:57.545 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 -- 172.21.15.33:0/2177604864 learned_addr learned my addr 172.21.15.33:0/2177604864 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee5a45700 1 --2- 172.21.15.33:0/2177604864 >> v2:172.21.15.167:3300/0 conn(0x7feee00b0530 0x7feee00b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6246700 1 --2- 172.21.15.33:0/2177604864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee00b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 -- 172.21.15.33:0/2177604864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 msgr2=0x7feee00b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 --2- 172.21.15.33:0/2177604864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee00b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 -- 172.21.15.33:0/2177604864 >> v2:172.21.15.167:3300/0 conn(0x7feee00b0530 msgr2=0x7feee00b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 --2- 172.21.15.33:0/2177604864 >> v2:172.21.15.167:3300/0 conn(0x7feee00b0530 0x7feee00b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 -- 172.21.15.33:0/2177604864 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00b00a0 con 0x7feee0158fc0 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee6a47700 1 --2- 172.21.15.33:0/2177604864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee0158fc0 0x7feee015ca70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feee0064430 tx=0x7feed8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8385bcd576ccbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee5244700 1 -- 172.21.15.33:0/2177604864 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed800e070 con 0x7feee0158fc0 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee5244700 1 -- 172.21.15.33:0/2177604864 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed800ba50 con 0x7feee0158fc0 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee5244700 1 -- 172.21.15.33:0/2177604864 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80092e0 con 0x7feee0158fc0 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- 172.21.15.33:0/2177604864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee0158fc0 msgr2=0x7feee015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- 172.21.15.33:0/2177604864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee0158fc0 0x7feee015ca70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feee0064430 tx=0x7feed8009840 comp rx=0 tx=0).stop 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- 172.21.15.33:0/2177604864 shutdown_connections 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- 172.21.15.33:0/2177604864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee0158fc0 0x7feee015ca70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- 172.21.15.33:0/2177604864 >> v2:172.21.15.167:3300/0 conn(0x7feee00b0530 0x7feee00b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 --2- 172.21.15.33:0/2177604864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee00b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- 172.21.15.33:0/2177604864 >> 172.21.15.33:0/2177604864 conn(0x7feee00cfd70 msgr2=0x7feee00d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- 172.21.15.33:0/2177604864 shutdown_connections 2023-10-06T02:46:57.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.541+0000 7feee84aa700 1 -- 172.21.15.33:0/2177604864 wait complete. 2023-10-06T02:46:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 Processor -- start 2023-10-06T02:46:57.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 -- start start 2023-10-06T02:46:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee016c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feee0158fc0 0x7feee01681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 0x7feee00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee00c4ba0 con 0x7feee00c46e0 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee00c4e80 con 0x7feee0158fc0 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee84aa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee00c5160 con 0x7feee0158bf0 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee6a47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 0x7feee00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee6a47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 0x7feee00cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59330/0 (socket says 172.21.15.33:59330) 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee6a47700 1 -- 172.21.15.33:0/639342406 learned_addr learned my addr 172.21.15.33:0/639342406 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee6246700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee016c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.545+0000 7feee5a45700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feee0158fc0 0x7feee01681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee6a47700 1 -- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 msgr2=0x7feee016c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee6a47700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee016c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee6a47700 1 -- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feee0158fc0 msgr2=0x7feee01681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee6a47700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feee0158fc0 0x7feee01681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee6a47700 1 -- 172.21.15.33:0/639342406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8009ae0 con 0x7feee00c46e0 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee6a47700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 0x7feee00cb9e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feee00694e0 tx=0x7feed8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80032f0 con 0x7feee00c46e0 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8008d20 con 0x7feee00c46e0 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee00c5440 con 0x7feee00c46e0 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8025690 con 0x7feee00c46e0 2023-10-06T02:46:57.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee00b4c90 con 0x7feee00c46e0 2023-10-06T02:46:57.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee0067490 con 0x7feee00c46e0 2023-10-06T02:46:57.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.549+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7feed8003040 con 0x7feee00c46e0 2023-10-06T02:46:57.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.553+0000 7feed27fc700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feec4060550 0x7feec4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.553+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7feed807e080 con 0x7feee00c46e0 2023-10-06T02:46:57.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.553+0000 7feee6246700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feec4060550 0x7feec4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.557 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.553+0000 7feee6246700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feec4060550 0x7feec4062a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feed4006fd0 tx=0x7feed4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.567 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.561+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feee0067490 con 0x7feee00c46e0 2023-10-06T02:46:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.725+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee0067490 con 0x7feee00c46e0 2023-10-06T02:46:57.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feed27fc700 1 -- 172.21.15.33:0/639342406 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17071 (secure 0 0 0) 0x7feee0067490 con 0x7feee00c46e0 2023-10-06T02:46:57.733 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:57.733 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":44,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:29.309782+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:46:25.764531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T02:46:26.918595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:57.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feec4060550 msgr2=0x7feec4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feec4060550 0x7feec4062a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feed4006fd0 tx=0x7feed4008040 comp rx=0 tx=0).stop 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 msgr2=0x7feee00cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 0x7feee00cb9e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feee00694e0 tx=0x7feed8018000 comp rx=0 tx=0).stop 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 shutdown_connections 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feee00c46e0 0x7feee00cb9e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feee0158fc0 0x7feee01681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7feec4060550 0x7feec4062a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 --2- 172.21.15.33:0/639342406 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feee0158bf0 0x7feee016c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 >> 172.21.15.33:0/639342406 conn(0x7feee00cfd70 msgr2=0x7feee00cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 shutdown_connections 2023-10-06T02:46:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.729+0000 7feee84aa700 1 -- 172.21.15.33:0/639342406 wait complete. 2023-10-06T02:46:57.748 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:46:57.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.845+0000 7fc7cbfff700 1 Processor -- start 2023-10-06T02:46:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.845+0000 7fc7cbfff700 1 -- start start 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc7cc06b050 0x7fc7cc066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc06b420 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc06a4e0 con 0x7fc7cc0c93a0 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc0674f0 con 0x7fc7cc06b050 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc7cc0c67e0 0x7fc7cc166c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57322/0 (socket says 172.21.15.33:57322) 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cb7fe700 1 -- 172.21.15.33:0/2688452520 learned_addr learned my addr 172.21.15.33:0/2688452520 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7caffd700 1 --2- 172.21.15.33:0/2688452520 >> v2:172.21.15.167:3300/0 conn(0x7fc7cc06b050 0x7fc7cc066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7c9ffb700 1 -- 172.21.15.33:0/2688452520 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825052709 0 0) 0x7fc7cc06b420 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7c9ffb700 1 -- 172.21.15.33:0/2688452520 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b4003570 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7caffd700 1 -- 172.21.15.33:0/2688452520 >> v1:172.21.15.167:6789/0 conn(0x7fc7cc0c93a0 legacy=0x7fc7cc0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7c9ffb700 1 -- 172.21.15.33:0/2688452520 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 912463489 0 0) 0x7fc7b4003570 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7caffd700 1 -- 172.21.15.33:0/2688452520 >> v1:172.21.15.33:6789/0 conn(0x7fc7cc0c67e0 legacy=0x7fc7cc166c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7caffd700 1 -- 172.21.15.33:0/2688452520 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc067700 con 0x7fc7cc06b050 2023-10-06T02:46:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7caffd700 1 --2- 172.21.15.33:0/2688452520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc066fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0098e0 tx=0x7fc7bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cec63650b31a495b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7c9ffb700 1 -- 172.21.15.33:0/2688452520 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40017f0 con 0x7fc7cc06b050 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7c9ffb700 1 -- 172.21.15.33:0/2688452520 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc011420 con 0x7fc7cc06b050 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7c9ffb700 1 -- 172.21.15.33:0/2688452520 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0117a0 con 0x7fc7cc06b050 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2688452520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 msgr2=0x7fc7cc066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2688452520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc066fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0098e0 tx=0x7fc7bc0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2688452520 shutdown_connections 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2688452520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc066fb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2688452520 >> 172.21.15.33:0/2688452520 conn(0x7fc7cc0c8240 msgr2=0x7fc7cc0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2688452520 shutdown_connections 2023-10-06T02:46:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.849+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2688452520 wait complete. 2023-10-06T02:46:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 Processor -- start 2023-10-06T02:46:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 -- start start 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7cc0c67e0 0x7fc7cc1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 0x7fc7cc0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15de40 con 0x7fc7cc0c93a0 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15dba0 con 0x7fc7cc06b050 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15f260 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 0x7fc7cc0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 0x7fc7cc0c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59332/0 (socket says 172.21.15.33:59332) 2023-10-06T02:46:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cb7fe700 1 -- 172.21.15.33:0/2676986935 learned_addr learned my addr 172.21.15.33:0/2676986935 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7cc0c67e0 0x7fc7cc1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7caffd700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 -- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 msgr2=0x7fc7cc15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc15d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 -- 172.21.15.33:0/2676986935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 msgr2=0x7fc7cc0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 0x7fc7cc0c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 -- 172.21.15.33:0/2676986935 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc009420 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cb7fe700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 0x7fc7cc0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:46:57.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7ca7fc700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7cc0c67e0 0x7fc7cc1595f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c590 tx=0x7fc7c0007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0011590 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0011710 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc15f540 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0011ad0 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.853+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc15fb00 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.857+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc071040 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.857+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc7c001b3b0 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.857+0000 7fc7b2ffd700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc7b8060580 0x7fc7b8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.857+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fc7c007bdc0 con 0x7fc7cc0c67e0 2023-10-06T02:46:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.857+0000 7fc7caffd700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc7b8060580 0x7fc7b8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:57.862 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.857+0000 7fc7caffd700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc7b8060580 0x7fc7b8062a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009fd0 tx=0x7fc7bc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:57.865+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc7c004c110 con 0x7fc7cc0c67e0 2023-10-06T02:46:58.068 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7cc0679e0 con 0x7fc7cc0c67e0 2023-10-06T02:46:58.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7b2ffd700 1 -- 172.21.15.33:0/2676986935 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fc7c004f730 con 0x7fc7cc0c67e0 2023-10-06T02:46:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:58.069 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc7b8060580 msgr2=0x7fc7b8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc7b8060580 0x7fc7b8062a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009fd0 tx=0x7fc7bc016000 comp rx=0 tx=0).stop 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7cc0c67e0 msgr2=0x7fc7cc1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7cc0c67e0 0x7fc7cc1595f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c590 tx=0x7fc7c0007ec0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 shutdown_connections 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc7b8060580 0x7fc7b8062a30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7cc0c67e0 0x7fc7cc1595f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7cc0c93a0 0x7fc7cc0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.065+0000 7fc7cbfff700 1 --2- 172.21.15.33:0/2676986935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc06b050 0x7fc7cc15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.069+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 >> 172.21.15.33:0/2676986935 conn(0x7fc7cc0c8240 msgr2=0x7fc7cc0c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:58.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.069+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 shutdown_connections 2023-10-06T02:46:58.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.069+0000 7fc7cbfff700 1 -- 172.21.15.33:0/2676986935 wait complete. 2023-10-06T02:46:58.073 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:58.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:46:58.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 Processor -- start 2023-10-06T02:46:58.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 -- start start 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 0x7f3e5c068450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e5c068990 0x7f3e5c15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c0cdf80 con 0x7f3e5c15cfb0 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c15da10 con 0x7f3e5c068990 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e61226700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c068de0 con 0x7f3e5c158d00 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e60a25700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3e5c15cfb0 0x7f3e5c0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57326/0 (socket says 172.21.15.33:57326) 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e60a25700 1 -- 172.21.15.33:0/1645288824 learned_addr learned my addr 172.21.15.33:0/1645288824 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:58.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5b7fe700 1 --2- 172.21.15.33:0/1645288824 >> v2:172.21.15.33:3300/0 conn(0x7f3e5c068990 0x7f3e5c15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 0x7f3e5c068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432277026 0 0) 0x7f3e5c0cdf80 con 0x7f3e5c15cfb0 2023-10-06T02:46:58.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e48003570 con 0x7f3e5c15cfb0 2023-10-06T02:46:58.195 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729630433 0 0) 0x7f3e48003570 con 0x7f3e5c15cfb0 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 msgr2=0x7f3e5c068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 0x7f3e5c068450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 >> v2:172.21.15.33:3300/0 conn(0x7f3e5c068990 msgr2=0x7f3e5c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 --2- 172.21.15.33:0/1645288824 >> v2:172.21.15.33:3300/0 conn(0x7f3e5c068990 0x7f3e5c15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c15d730 con 0x7f3e5c15cfb0 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f3e540017f0 con 0x7f3e5c15cfb0 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 >> v1:172.21.15.33:6789/0 conn(0x7f3e5c15cfb0 legacy=0x7f3e5c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48003a60 0x7f3e48003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e48004810 0x7f3e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 0x7f3e480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15da10 con 0x7f3e48003a60 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c068de0 con 0x7f3e48004810 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4800bae0 con 0x7f3e5c158d00 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48003a60 0x7f3e48003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5b7fe700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e48004810 0x7f3e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e60a25700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 0x7f3e480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 -- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 msgr2=0x7f3e480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c158d00 0x7f3e480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 -- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e48004810 msgr2=0x7f3e48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e48004810 0x7f3e48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 -- 172.21.15.33:0/1645288824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48003570 con 0x7f3e48003a60 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48003a60 0x7f3e48003e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009c00 tx=0x7f3e4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492bd77215ecee9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00e710 con 0x7f3e48003a60 2023-10-06T02:46:58.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c00e870 con 0x7f3e48003a60 2023-10-06T02:46:58.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.189+0000 7f3e5affd700 1 -- 172.21.15.33:0/1645288824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00ec30 con 0x7f3e48003a60 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- 172.21.15.33:0/1645288824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48003a60 msgr2=0x7f3e48003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 --2- 172.21.15.33:0/1645288824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e48003a60 0x7f3e48003e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009c00 tx=0x7f3e4c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e60a25700 1 -- 172.21.15.33:0/1645288824 reap_dead start 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- 172.21.15.33:0/1645288824 shutdown_connections 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- 172.21.15.33:0/1645288824 >> 172.21.15.33:0/1645288824 conn(0x7f3e5c067a80 msgr2=0x7f3e5c0cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- 172.21.15.33:0/1645288824 shutdown_connections 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- 172.21.15.33:0/1645288824 wait complete. 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 Processor -- start 2023-10-06T02:46:58.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- start start 2023-10-06T02:46:58.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 0x7f3e5c0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5c0c9380 0x7f3e5c0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c0c6090 0x7f3e5c16c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1692f0 con 0x7f3e5c147930 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c6500 con 0x7f3e5c0c9380 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.193+0000 7f3e61226700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b3d60 con 0x7f3e5c0c6090 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 0x7f3e5c0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 0x7f3e5c0c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59348/0 (socket says 172.21.15.33:59348) 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 -- 172.21.15.33:0/920788131 learned_addr learned my addr 172.21.15.33:0/920788131 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5b7fe700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5c0c9380 0x7f3e5c0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e60a25700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c0c6090 0x7f3e5c16c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 -- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c0c6090 msgr2=0x7f3e5c16c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c0c6090 0x7f3e5c16c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 -- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5c0c9380 msgr2=0x7f3e5c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5c0c9380 0x7f3e5c0c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 -- 172.21.15.33:0/920788131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0b4040 con 0x7f3e5c147930 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e5bfff700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 0x7f3e5c0c8e40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3e480057d0 tx=0x7f3e4c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c022070 con 0x7f3e5c147930 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c00e9e0 con 0x7f3e5c147930 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c0b4320 con 0x7f3e5c147930 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00e710 con 0x7f3e5c147930 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c0b39a0 con 0x7f3e5c147930 2023-10-06T02:46:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.197+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c0cb4a0 con 0x7f3e5c147930 2023-10-06T02:46:58.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.201+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3e4c01c660 con 0x7f3e5c147930 2023-10-06T02:46:58.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.201+0000 7f3e53fff700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e3c060550 0x7f3e3c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.201+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3e4c07df50 con 0x7f3e5c147930 2023-10-06T02:46:58.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.201+0000 7f3e5b7fe700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e3c060550 0x7f3e3c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.205 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.201+0000 7f3e5b7fe700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e3c060550 0x7f3e3c062a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0cd510 tx=0x7f3e440091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.205+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3e5c0cb4a0 con 0x7f3e5c147930 2023-10-06T02:46:58.408 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.405+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e5c0cb4a0 con 0x7f3e5c147930 2023-10-06T02:46:58.409 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.405+0000 7f3e53fff700 1 -- 172.21.15.33:0/920788131 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f3e5c0cb4a0 con 0x7f3e5c147930 2023-10-06T02:46:58.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:58.410 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T02:46:27.926852+0000","modified":"2023-10-06T02:46:30.566783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5145":{"gid":5145,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/3594813995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3594813995},{"type":"v1","addr":"172.21.15.33:6835","nonce":3594813995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e3c060550 msgr2=0x7f3e3c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e3c060550 0x7f3e3c062a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0cd510 tx=0x7f3e440091a0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 msgr2=0x7f3e5c0c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 0x7f3e5c0c8e40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3e480057d0 tx=0x7f3e4c01a000 comp rx=0 tx=0).stop 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 shutdown_connections 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3e3c060550 0x7f3e3c062a00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e5c0c6090 0x7f3e5c16c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e5c0c9380 0x7f3e5c0c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 --2- 172.21.15.33:0/920788131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c147930 0x7f3e5c0c8e40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 >> 172.21.15.33:0/920788131 conn(0x7f3e5c067a80 msgr2=0x7f3e5c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 shutdown_connections 2023-10-06T02:46:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.409+0000 7f3e61226700 1 -- 172.21.15.33:0/920788131 wait complete. 2023-10-06T02:46:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2023-10-06T02:46:58.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 Processor -- start 2023-10-06T02:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 -- start start 2023-10-06T02:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef00cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ef00c3ab0 0x7f9ef00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef00b0460 con 0x7f9ef00cc410 2023-10-06T02:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00cc9b0 con 0x7f9ef00c3ab0 2023-10-06T02:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eef59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00ccc90 con 0x7f9ef00b0090 2023-10-06T02:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ef00c3ab0 0x7f9ef00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ef00c3ab0 0x7f9ef00c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59364/0 (socket says 172.21.15.33:59364) 2023-10-06T02:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eee59c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef00cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 -- 172.21.15.33:0/297204730 learned_addr learned my addr 172.21.15.33:0/297204730 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eedd9b700 1 --1- 172.21.15.33:0/297204730 >> v1:172.21.15.167:6789/0 conn(0x7f9ef00cc410 0x7f9ef00c33a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:60046/0 (socket says 172.21.15.33:60046) 2023-10-06T02:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 -- 172.21.15.33:0/297204730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 msgr2=0x7f9ef00cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 --2- 172.21.15.33:0/297204730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef00cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 -- 172.21.15.33:0/297204730 >> v1:172.21.15.167:6789/0 conn(0x7f9ef00cc410 legacy=0x7f9ef00c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 -- 172.21.15.33:0/297204730 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00ccf70 con 0x7f9ef00c3ab0 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eed59a700 1 -- 172.21.15.33:0/297204730 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297535482 0 0) 0x7f9ef00b0460 con 0x7f9ef00cc410 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eeed9d700 1 --2- 172.21.15.33:0/297204730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef00c8290 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80099b0 tx=0x7f9ed8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c653013bced182 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eed59a700 1 -- 172.21.15.33:0/297204730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8009ea0 con 0x7f9ef00c3ab0 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eed59a700 1 -- 172.21.15.33:0/297204730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed800b860 con 0x7f9ef00c3ab0 2023-10-06T02:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.525+0000 7f9eed59a700 1 -- 172.21.15.33:0/297204730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800bc00 con 0x7f9ef00c3ab0 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 -- 172.21.15.33:0/297204730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00c3ab0 msgr2=0x7f9ef00c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 --2- 172.21.15.33:0/297204730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef00c8290 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80099b0 tx=0x7f9ed8009250 comp rx=0 tx=0).stop 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 -- 172.21.15.33:0/297204730 shutdown_connections 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 --2- 172.21.15.33:0/297204730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef00c8290 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 --2- 172.21.15.33:0/297204730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef00cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 -- 172.21.15.33:0/297204730 >> 172.21.15.33:0/297204730 conn(0x7f9ef0086ff0 msgr2=0x7f9ef0096cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 -- 172.21.15.33:0/297204730 shutdown_connections 2023-10-06T02:46:58.534 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 -- 172.21.15.33:0/297204730 wait complete. 2023-10-06T02:46:58.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.529+0000 7f9eef59e700 1 Processor -- start 2023-10-06T02:46:58.535 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 -- start start 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef016d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef0150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 0x7f9ef00680a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef016d590 con 0x7f9ef00cc410 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00b2800 con 0x7f9ef00c3ab0 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00b1bf0 con 0x7f9ef00b0090 2023-10-06T02:46:58.538 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 0x7f9ef00680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 0x7f9ef00680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59372/0 (socket says 172.21.15.33:59372) 2023-10-06T02:46:58.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 -- 172.21.15.33:0/2177403545 learned_addr learned my addr 172.21.15.33:0/2177403545 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:58.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eee59c700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef016d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.539 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eedd9b700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef0150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 -- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 msgr2=0x7f9ef016d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef016d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 -- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00c3ab0 msgr2=0x7f9ef0150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef0150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 -- 172.21.15.33:0/2177403545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed80094f0 con 0x7f9ef00cc410 2023-10-06T02:46:58.540 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eeed9d700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 0x7f9ef00680a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80099b0 tx=0x7f9ed8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800bbe0 con 0x7f9ef00cc410 2023-10-06T02:46:58.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8011420 con 0x7f9ef00cc410 2023-10-06T02:46:58.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef00b1ed0 con 0x7f9ef00cc410 2023-10-06T02:46:58.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80117c0 con 0x7f9ef00cc410 2023-10-06T02:46:58.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.533+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef00b2490 con 0x7f9ef00cc410 2023-10-06T02:46:58.541 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.537+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef00c5ea0 con 0x7f9ef00cc410 2023-10-06T02:46:58.542 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.537+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9ed8011bb0 con 0x7f9ef00cc410 2023-10-06T02:46:58.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.537+0000 7f9ee67fc700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9ed4060580 0x7f9ed4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.537+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f9ed807da80 con 0x7f9ef00cc410 2023-10-06T02:46:58.543 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.537+0000 7f9eee59c700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9ed4060580 0x7f9ed4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.541+0000 7f9eee59c700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9ed4060580 0x7f9ed4062a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800f510 tx=0x7f9ee800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.545+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9ef00c5ea0 con 0x7f9ef00cc410 2023-10-06T02:46:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.709+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ef00c5ea0 con 0x7f9ef00cc410 2023-10-06T02:46:58.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.713+0000 7f9ee67fc700 1 -- 172.21.15.33:0/2177403545 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17071 (secure 0 0 0) 0x7f9ef00c5ea0 con 0x7f9ef00cc410 2023-10-06T02:46:58.718 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:46:58.718 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":44,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:46:29.309782+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-06T02:46:25.764531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-06T02:46:26.918595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:46:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9ed4060580 msgr2=0x7f9ed4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9ed4060580 0x7f9ed4062a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800f510 tx=0x7f9ee800b040 comp rx=0 tx=0).stop 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 msgr2=0x7f9ef00680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 0x7f9ef00680a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80099b0 tx=0x7f9ed8017000 comp rx=0 tx=0).stop 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 shutdown_connections 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef00cc410 0x7f9ef00680a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ef00c3ab0 0x7f9ef0150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f9ed4060580 0x7f9ed4062a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 --2- 172.21.15.33:0/2177403545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ef00b0090 0x7f9ef016d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 >> 172.21.15.33:0/2177403545 conn(0x7f9ef0086ff0 msgr2=0x7f9ef00ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 shutdown_connections 2023-10-06T02:46:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.717+0000 7f9eef59e700 1 -- 172.21.15.33:0/2177403545 wait complete. 2023-10-06T02:46:58.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T02:46:58.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.837+0000 7f46ddd35700 1 Processor -- start 2023-10-06T02:46:58.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.837+0000 7f46ddd35700 1 -- start start 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f46d80c6b40 0x7f46d8155260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d80c7d30 con 0x7f46d8154e90 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d80c1a90 con 0x7f46d8067840 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80c81c0 con 0x7f46d80c6b40 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f46d8154e90 0x7f46d80c1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57338/0 (socket says 172.21.15.33:57338) 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d77fe700 1 -- 172.21.15.33:0/1951471696 learned_addr learned my addr 172.21.15.33:0/1951471696 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:58.844 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d6ffd700 1 --2- 172.21.15.33:0/1951471696 >> v2:172.21.15.167:3300/0 conn(0x7f46d80c6b40 0x7f46d8155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d67fc700 1 -- 172.21.15.33:0/1951471696 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45942000 0 0) 0x7f46d80c7d30 con 0x7f46d8154e90 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d67fc700 1 -- 172.21.15.33:0/1951471696 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c4003570 con 0x7f46d8154e90 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d6ffd700 1 -- 172.21.15.33:0/1951471696 >> v1:172.21.15.167:6789/0 conn(0x7f46d8067840 legacy=0x7f46d80c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d67fc700 1 -- 172.21.15.33:0/1951471696 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505031785 0 0) 0x7f46d80c1a90 con 0x7f46d8067840 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d6ffd700 1 -- 172.21.15.33:0/1951471696 >> v1:172.21.15.33:6789/0 conn(0x7f46d8154e90 legacy=0x7f46d80c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d6ffd700 1 -- 172.21.15.33:0/1951471696 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80c7120 con 0x7f46d80c6b40 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d6ffd700 1 --2- 172.21.15.33:0/1951471696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d80c6b40 0x7f46d8155260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009e60 tx=0x7f46c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c542c6b03337b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d67fc700 1 -- 172.21.15.33:0/1951471696 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000e070 con 0x7f46d80c6b40 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d67fc700 1 -- 172.21.15.33:0/1951471696 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000ba60 con 0x7f46d80c6b40 2023-10-06T02:46:58.846 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46d67fc700 1 -- 172.21.15.33:0/1951471696 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000be00 con 0x7f46d80c6b40 2023-10-06T02:46:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- 172.21.15.33:0/1951471696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d80c6b40 msgr2=0x7f46d8155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 --2- 172.21.15.33:0/1951471696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d80c6b40 0x7f46d8155260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009e60 tx=0x7f46c0009780 comp rx=0 tx=0).stop 2023-10-06T02:46:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- 172.21.15.33:0/1951471696 shutdown_connections 2023-10-06T02:46:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 --2- 172.21.15.33:0/1951471696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d80c6b40 0x7f46d8155260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- 172.21.15.33:0/1951471696 >> 172.21.15.33:0/1951471696 conn(0x7f46d80c83f0 msgr2=0x7f46d80c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- 172.21.15.33:0/1951471696 shutdown_connections 2023-10-06T02:46:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.841+0000 7f46ddd35700 1 -- 172.21.15.33:0/1951471696 wait complete. 2023-10-06T02:46:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 Processor -- start 2023-10-06T02:46:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 -- start start 2023-10-06T02:46:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 0x7f46d8146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d80c6b40 0x7f46d8146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d8154e90 0x7f46d8069870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8148500 con 0x7f46d8067840 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8147880 con 0x7f46d8154e90 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d8147b60 con 0x7f46d80c6b40 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 0x7f46d8146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 0x7f46d8146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59374/0 (socket says 172.21.15.33:59374) 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 -- 172.21.15.33:0/4095722039 learned_addr learned my addr 172.21.15.33:0/4095722039 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d6ffd700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d80c6b40 0x7f46d8146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d7fff700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d8154e90 0x7f46d8069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 -- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d80c6b40 msgr2=0x7f46d8146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d80c6b40 0x7f46d8146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 -- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d8154e90 msgr2=0x7f46d8069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d8154e90 0x7f46d8069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 -- 172.21.15.33:0/4095722039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009a20 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46d77fe700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 0x7f46d8146650 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c810 tx=0x7f46c8007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8007360 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80074e0 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d8147e40 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004830 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.845+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d8146b90 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.849+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d80710b0 con 0x7f46d8067840 2023-10-06T02:46:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.849+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f46c8004bc0 con 0x7f46d8067840 2023-10-06T02:46:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.849+0000 7f46cf7fe700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060580 0x7f46b8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:58.854 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.849+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f46c807c040 con 0x7f46d8067840 2023-10-06T02:46:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.849+0000 7f46d6ffd700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060580 0x7f46b8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.853+0000 7f46d6ffd700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060580 0x7f46b8062a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099f0 tx=0x7f46c0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:58.853+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f46c804c370 con 0x7f46d8067840 2023-10-06T02:46:59.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.049+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f46d8068700 con 0x7f46d8067840 2023-10-06T02:46:59.054 INFO:tasks.ceph.mon.a.smithi033.stderr:2023-10-06T02:46:59.049+0000 7f7675d7e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T02:46:59.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.057+0000 7f46cf7fe700 1 -- 172.21.15.33:0/4095722039 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f46c8018090 con 0x7f46d8067840 2023-10-06T02:46:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060580 msgr2=0x7f46b8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060580 0x7f46b8062a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099f0 tx=0x7f46c0017000 comp rx=0 tx=0).stop 2023-10-06T02:46:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 msgr2=0x7f46d8146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 0x7f46d8146650 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c810 tx=0x7f46c8007d20 comp rx=0 tx=0).stop 2023-10-06T02:46:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 shutdown_connections 2023-10-06T02:46:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f46b8060580 0x7f46b8062a30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46d80c6b40 0x7f46d8146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46d8154e90 0x7f46d8069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.065 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 --2- 172.21.15.33:0/4095722039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d8067840 0x7f46d8146650 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 >> 172.21.15.33:0/4095722039 conn(0x7f46d80c83f0 msgr2=0x7f46d80ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 shutdown_connections 2023-10-06T02:46:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.061+0000 7f46ddd35700 1 -- 172.21.15.33:0/4095722039 wait complete. 2023-10-06T02:46:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T02:46:59.068 INFO:tasks.ceph.mds.a.smithi033.stderr: -5237> 2023-10-06T02:46:31.489+0000 7efbf94db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T02:46:59.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T02:46:59.097 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2023-10-06T02:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.181+0000 7faae0fee700 1 Processor -- start 2023-10-06T02:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.181+0000 7faae0fee700 1 -- start start 2023-10-06T02:46:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7faadc0c5e30 0x7faadc0ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faadc0c6b10 con 0x7faadc0bee30 2023-10-06T02:46:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faadc0c8a10 con 0x7faadc06a570 2023-10-06T02:46:59.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faadc0c6910 con 0x7faadc0c5e30 2023-10-06T02:46:59.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadaffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faadc0bee30 0x7faadc0c6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57354/0 (socket says 172.21.15.33:57354) 2023-10-06T02:46:59.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadaffd700 1 -- 172.21.15.33:0/2259905061 learned_addr learned my addr 172.21.15.33:0/2259905061 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:46:59.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadb7fe700 1 --2- 172.21.15.33:0/2259905061 >> v2:172.21.15.167:3300/0 conn(0x7faadc0c5e30 0x7faadc0ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faada7fc700 1 -- 172.21.15.33:0/2259905061 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646189576 0 0) 0x7faadc0c6b10 con 0x7faadc0bee30 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faada7fc700 1 -- 172.21.15.33:0/2259905061 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faac4003570 con 0x7faadc0bee30 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadb7fe700 1 -- 172.21.15.33:0/2259905061 >> v1:172.21.15.167:6789/0 conn(0x7faadc06a570 legacy=0x7faadc067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadb7fe700 1 -- 172.21.15.33:0/2259905061 >> v1:172.21.15.33:6789/0 conn(0x7faadc0bee30 legacy=0x7faadc0c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadb7fe700 1 -- 172.21.15.33:0/2259905061 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0c8cf0 con 0x7faadc0c5e30 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faadb7fe700 1 --2- 172.21.15.33:0/2259905061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc0ab4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faad4009e60 tx=0x7faad4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73973f1ccf05d432 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faada7fc700 1 -- 172.21.15.33:0/2259905061 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad400e070 con 0x7faadc0c5e30 2023-10-06T02:46:59.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faada7fc700 1 -- 172.21.15.33:0/2259905061 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad400bb60 con 0x7faadc0c5e30 2023-10-06T02:46:59.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faada7fc700 1 -- 172.21.15.33:0/2259905061 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4012430 con 0x7faadc0c5e30 2023-10-06T02:46:59.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- 172.21.15.33:0/2259905061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 msgr2=0x7faadc0ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 --2- 172.21.15.33:0/2259905061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc0ab4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faad4009e60 tx=0x7faad4009780 comp rx=0 tx=0).stop 2023-10-06T02:46:59.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- 172.21.15.33:0/2259905061 shutdown_connections 2023-10-06T02:46:59.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 --2- 172.21.15.33:0/2259905061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc0ab4e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- 172.21.15.33:0/2259905061 >> 172.21.15.33:0/2259905061 conn(0x7faadc0cc2c0 msgr2=0x7faadc0c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:46:59.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- 172.21.15.33:0/2259905061 shutdown_connections 2023-10-06T02:46:59.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.185+0000 7faae0fee700 1 -- 172.21.15.33:0/2259905061 wait complete. 2023-10-06T02:46:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.189+0000 7faae0fee700 1 Processor -- start 2023-10-06T02:46:59.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.189+0000 7faae0fee700 1 -- start start 2023-10-06T02:46:59.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faadc06a570 0x7faadc160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 0x7faadc0c1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0c33e0 con 0x7faadc0bee30 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0c36c0 con 0x7faadc0c5e30 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0c0c90 con 0x7faadc06a570 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 0x7faadc0c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 0x7faadc0c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59398/0 (socket says 172.21.15.33:59398) 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 -- 172.21.15.33:0/3034418848 learned_addr learned my addr 172.21.15.33:0/3034418848 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadb7fe700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faadc06a570 0x7faadc160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadbfff700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 -- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faadc06a570 msgr2=0x7faadc160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faadc06a570 0x7faadc160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 -- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 msgr2=0x7faadc15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 -- 172.21.15.33:0/3034418848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad4009a20 con 0x7faadc0bee30 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadbfff700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faadaffd700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 0x7faadc0c1770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faad000c7c0 tx=0x7faad0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0005b30 con 0x7faadc0bee30 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad0005c90 con 0x7faadc0bee30 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc0c0f70 con 0x7faadc0bee30 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0007340 con 0x7faadc0bee30 2023-10-06T02:46:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc15f970 con 0x7faadc0bee30 2023-10-06T02:46:59.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc0c1470 con 0x7faadc0bee30 2023-10-06T02:46:59.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.193+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faad001b400 con 0x7faadc0bee30 2023-10-06T02:46:59.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.197+0000 7faac37fe700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faac8060580 0x7faac8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:46:59.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.197+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6464+0+0 (secure 0 0 0) 0x7faad007c550 con 0x7faadc0bee30 2023-10-06T02:46:59.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.197+0000 7faadb7fe700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faac8060580 0x7faac8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:46:59.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.197+0000 7faadb7fe700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faac8060580 0x7faac8062a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad40097c0 tx=0x7faad4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:46:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.201+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faad004c8d0 con 0x7faadc0bee30 2023-10-06T02:46:59.400 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:46:59.397+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faadc071040 con 0x7faadc0bee30 2023-10-06T02:47:00.066 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.060+0000 7faac37fe700 1 -- 172.21.15.33:0/3034418848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7faad004fef0 con 0x7faadc0bee30 2023-10-06T02:47:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faac8060580 msgr2=0x7faac8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faac8060580 0x7faac8062a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad40097c0 tx=0x7faad4019000 comp rx=0 tx=0).stop 2023-10-06T02:47:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 msgr2=0x7faadc0c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 0x7faadc0c1770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faad000c7c0 tx=0x7faad0007e00 comp rx=0 tx=0).stop 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 shutdown_connections 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faadc0c5e30 0x7faadc15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faadc0bee30 0x7faadc0c1770 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7faac8060580 0x7faac8062a30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 --2- 172.21.15.33:0/3034418848 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faadc06a570 0x7faadc160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 >> 172.21.15.33:0/3034418848 conn(0x7faadc0cc2c0 msgr2=0x7faadc0c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 shutdown_connections 2023-10-06T02:47:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.064+0000 7faae0fee700 1 -- 172.21.15.33:0/3034418848 wait complete. 2023-10-06T02:47:00.089 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T02:47:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.248+0000 7f6aa5e67700 1 Processor -- start 2023-10-06T02:47:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.248+0000 7f6aa5e67700 1 -- start start 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5e67700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6aa00b18e0 0x7f6aa00b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5e67700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa0069f30 con 0x7f6aa015bde0 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5e67700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa0069320 con 0x7f6aa015c1b0 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5e67700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa015c630 con 0x7f6aa00b18e0 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa4e65700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6aa015bde0 0x7f6aa0068a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57366/0 (socket says 172.21.15.33:57366) 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa4e65700 1 -- 172.21.15.33:0/1768115662 learned_addr learned my addr 172.21.15.33:0/1768115662 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5666700 1 --2- 172.21.15.33:0/1768115662 >> v2:172.21.15.167:3300/0 conn(0x7f6aa00b18e0 0x7f6aa00b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6a9f7fe700 1 -- 172.21.15.33:0/1768115662 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651659964 0 0) 0x7f6aa0069f30 con 0x7f6aa015bde0 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5666700 1 -- 172.21.15.33:0/1768115662 >> v1:172.21.15.167:6789/0 conn(0x7f6aa015c1b0 legacy=0x7f6aa00b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5666700 1 -- 172.21.15.33:0/1768115662 >> v1:172.21.15.33:6789/0 conn(0x7f6aa015bde0 legacy=0x7f6aa0068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5666700 1 -- 172.21.15.33:0/1768115662 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0069600 con 0x7f6aa00b18e0 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6aa5666700 1 --2- 172.21.15.33:0/1768115662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00b32a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009e60 tx=0x7f6a98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e9a11d68fac4952 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6a9f7fe700 1 -- 172.21.15.33:0/1768115662 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800e070 con 0x7f6aa00b18e0 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6a9f7fe700 1 -- 172.21.15.33:0/1768115662 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9800bc30 con 0x7f6aa00b18e0 2023-10-06T02:47:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.252+0000 7f6a9f7fe700 1 -- 172.21.15.33:0/1768115662 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98012430 con 0x7f6aa00b18e0 2023-10-06T02:47:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 -- 172.21.15.33:0/1768115662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa00b18e0 msgr2=0x7f6aa00b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/1768115662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00b32a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009e60 tx=0x7f6a98009780 comp rx=0 tx=0).stop 2023-10-06T02:47:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 -- 172.21.15.33:0/1768115662 shutdown_connections 2023-10-06T02:47:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/1768115662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00b32a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 -- 172.21.15.33:0/1768115662 >> 172.21.15.33:0/1768115662 conn(0x7f6aa006c310 msgr2=0x7f6aa006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 -- 172.21.15.33:0/1768115662 shutdown_connections 2023-10-06T02:47:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 -- 172.21.15.33:0/1768115662 wait complete. 2023-10-06T02:47:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 Processor -- start 2023-10-06T02:47:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.256+0000 7f6aa5e67700 1 -- start start 2023-10-06T02:47:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aa015bde0 0x7f6aa00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa015c1b0 0x7f6aa0144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00cc310 con 0x7f6aa00b18e0 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00cb700 con 0x7f6aa015c1b0 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00cb9e0 con 0x7f6aa015bde0 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa4e65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aa015bde0 0x7f6aa00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5666700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa015c1b0 0x7f6aa0144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa4e65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59400/0 (socket says 172.21.15.33:59400) 2023-10-06T02:47:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa4e65700 1 -- 172.21.15.33:0/4096413642 learned_addr learned my addr 172.21.15.33:0/4096413642 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 -- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa015c1b0 msgr2=0x7f6aa0144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa015c1b0 0x7f6aa0144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 -- 172.21.15.33:0/4096413642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa00b18e0 msgr2=0x7f6aa00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 -- 172.21.15.33:0/4096413642 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98009a20 con 0x7f6aa015bde0 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9ffff700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aa015bde0 0x7f6aa00cc7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c590 tx=0x7f6a8c007e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c011590 con 0x7f6aa015bde0 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c011710 con 0x7f6aa015bde0 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa00cbcc0 con 0x7f6aa015bde0 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c011ad0 con 0x7f6aa015bde0 2023-10-06T02:47:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.260+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa0143b80 con 0x7f6aa015bde0 2023-10-06T02:47:00.269 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.264+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6a8c01b3b0 con 0x7f6aa015bde0 2023-10-06T02:47:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.264+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa00b2920 con 0x7f6aa015bde0 2023-10-06T02:47:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.264+0000 7f6a9cff9700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6a84060580 0x7f6a84062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.264+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f6a8c07bd80 con 0x7f6aa015bde0 2023-10-06T02:47:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.264+0000 7f6aa4e65700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6a84060580 0x7f6a84062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.264+0000 7f6aa4e65700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6a84060580 0x7f6a84062a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a94007fd0 tx=0x7f6a94007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:00.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.272+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6aa00b2920 con 0x7f6aa015bde0 2023-10-06T02:47:00.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:00.440+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6aa00b2920 con 0x7f6aa015bde0 2023-10-06T02:47:01.437 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.432+0000 7f6a9cff9700 1 -- 172.21.15.33:0/4096413642 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6aa00b2920 con 0x7f6aa015bde0 2023-10-06T02:47:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6a84060580 msgr2=0x7f6a84062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6a84060580 0x7f6a84062a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a94007fd0 tx=0x7f6a94007df0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aa015bde0 msgr2=0x7f6aa00cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aa015bde0 0x7f6aa00cc7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c590 tx=0x7f6a8c007e80 comp rx=0 tx=0).stop 2023-10-06T02:47:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 shutdown_connections 2023-10-06T02:47:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6a84060580 0x7f6a84062a30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6aa015bde0 0x7f6aa00cc7a0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c590 tx=0x7f6a8c007e80 comp rx=0 tx=0).stop 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aa015c1b0 0x7f6aa0144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 --2- 172.21.15.33:0/4096413642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aa00b18e0 0x7f6aa00cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 >> 172.21.15.33:0/4096413642 conn(0x7f6aa006c310 msgr2=0x7f6aa00b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 shutdown_connections 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.436+0000 7f6aa5e67700 1 -- 172.21.15.33:0/4096413642 wait complete. 2023-10-06T02:47:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2023-10-06T02:47:01.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T02:47:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.560+0000 7efec695b700 1 Processor -- start 2023-10-06T02:47:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.560+0000 7efec695b700 1 -- start start 2023-10-06T02:47:01.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec015c3b0 0x7efec0159590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:01.568 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec015af10 con 0x7efec0068ba0 2023-10-06T02:47:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec0159fe0 con 0x7efec0158ab0 2023-10-06T02:47:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec015a8c0 con 0x7efec015c3b0 2023-10-06T02:47:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec5158700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efec0068ba0 0x7efec0158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57380/0 (socket says 172.21.15.33:57380) 2023-10-06T02:47:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec5158700 1 -- 172.21.15.33:0/2761515964 learned_addr learned my addr 172.21.15.33:0/2761515964 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec615a700 1 --2- 172.21.15.33:0/2761515964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec015c3b0 0x7efec0159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178656219 0 0) 0x7efec015af10 con 0x7efec0068ba0 2023-10-06T02:47:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeac003570 con 0x7efec0068ba0 2023-10-06T02:47:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258030451 0 0) 0x7efec0159fe0 con 0x7efec0158ab0 2023-10-06T02:47:01.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec015af10 con 0x7efec0158ab0 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789066851 0 0) 0x7efeac003570 con 0x7efec0068ba0 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec615a700 1 -- 172.21.15.33:0/2761515964 >> v1:172.21.15.167:6789/0 conn(0x7efec0158ab0 legacy=0x7efec0068490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec615a700 1 -- 172.21.15.33:0/2761515964 >> v1:172.21.15.33:6789/0 conn(0x7efec0068ba0 legacy=0x7efec0158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec615a700 1 -- 172.21.15.33:0/2761515964 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec015aba0 con 0x7efec015c3b0 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec615a700 1 --2- 172.21.15.33:0/2761515964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec015c3b0 0x7efec0159590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009910 tx=0x7efeb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb5721fb9d9be67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0001470 con 0x7efec015c3b0 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb8011420 con 0x7efec015c3b0 2023-10-06T02:47:01.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec4957700 1 -- 172.21.15.33:0/2761515964 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb80117c0 con 0x7efec015c3b0 2023-10-06T02:47:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 -- 172.21.15.33:0/2761515964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec015c3b0 msgr2=0x7efec0159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 --2- 172.21.15.33:0/2761515964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec015c3b0 0x7efec0159590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009910 tx=0x7efeb80091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 -- 172.21.15.33:0/2761515964 shutdown_connections 2023-10-06T02:47:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 --2- 172.21.15.33:0/2761515964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec015c3b0 0x7efec0159590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.564+0000 7efec695b700 1 -- 172.21.15.33:0/2761515964 >> 172.21.15.33:0/2761515964 conn(0x7efec0067a80 msgr2=0x7efec00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:01.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.568+0000 7efec695b700 1 -- 172.21.15.33:0/2761515964 shutdown_connections 2023-10-06T02:47:01.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.568+0000 7efec695b700 1 -- 172.21.15.33:0/2761515964 wait complete. 2023-10-06T02:47:01.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.568+0000 7efec695b700 1 Processor -- start 2023-10-06T02:47:01.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.568+0000 7efec695b700 1 -- start start 2023-10-06T02:47:01.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec0068ba0 0x7efec0153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 0x7efec00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec00b4750 0x7efec00ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec0153dd0 con 0x7efec0158ab0 2023-10-06T02:47:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00b4d80 con 0x7efec0068ba0 2023-10-06T02:47:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec00b5060 con 0x7efec00b4750 2023-10-06T02:47:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 0x7efec00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 0x7efec00b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59414/0 (socket says 172.21.15.33:59414) 2023-10-06T02:47:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 -- 172.21.15.33:0/1364216098 learned_addr learned my addr 172.21.15.33:0/1364216098 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec615a700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec00b4750 0x7efec00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5959700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec0068ba0 0x7efec0153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 -- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec00b4750 msgr2=0x7efec00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec00b4750 0x7efec00ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 -- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec0068ba0 msgr2=0x7efec0153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec0068ba0 0x7efec0153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 -- 172.21.15.33:0/1364216098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8009420 con 0x7efec0158ab0 2023-10-06T02:47:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec5158700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 0x7efec00b4210 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efeb000cb70 tx=0x7efeb0001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000ce80 con 0x7efec0158ab0 2023-10-06T02:47:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0007360 con 0x7efec0158ab0 2023-10-06T02:47:01.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec00b5340 con 0x7efec0158ab0 2023-10-06T02:47:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0007700 con 0x7efec0158ab0 2023-10-06T02:47:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec00b5900 con 0x7efec0158ab0 2023-10-06T02:47:01.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.572+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec00b4b50 con 0x7efec0158ab0 2023-10-06T02:47:01.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.576+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efeb0004a40 con 0x7efec0158ab0 2023-10-06T02:47:01.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.576+0000 7efeb5ffb700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efea4060550 0x7efea4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:01.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.576+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6026+0+0 (secure 0 0 0) 0x7efeb007c100 con 0x7efec0158ab0 2023-10-06T02:47:01.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.576+0000 7efec5959700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efea4060550 0x7efea4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:01.585 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.576+0000 7efec5959700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efea4060550 0x7efea4062a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efebc004140 tx=0x7efebc007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:01.591 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.584+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efeb004c590 con 0x7efec0158ab0 2023-10-06T02:47:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:01.752+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efec0153f10 con 0x7efec0158ab0 2023-10-06T02:47:02.442 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.436+0000 7efeb5ffb700 1 -- 172.21.15.33:0/1364216098 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7efeb004fbb0 con 0x7efec0158ab0 2023-10-06T02:47:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efea4060550 msgr2=0x7efea4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efea4060550 0x7efea4062a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efebc004140 tx=0x7efebc007f90 comp rx=0 tx=0).stop 2023-10-06T02:47:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 msgr2=0x7efec00b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 0x7efec00b4210 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efeb000cb70 tx=0x7efeb0001b50 comp rx=0 tx=0).stop 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 shutdown_connections 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7efea4060550 0x7efea4062a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efec00b4750 0x7efec00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efec0158ab0 0x7efec00b4210 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 --2- 172.21.15.33:0/1364216098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec0068ba0 0x7efec0153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 >> 172.21.15.33:0/1364216098 conn(0x7efec0067a80 msgr2=0x7efec0155950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 shutdown_connections 2023-10-06T02:47:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.440+0000 7efec695b700 1 -- 172.21.15.33:0/1364216098 wait complete. 2023-10-06T02:47:02.448 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2023-10-06T02:47:02.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T02:47:02.570 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.564+0000 7fd9e0c08700 1 Processor -- start 2023-10-06T02:47:02.571 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.564+0000 7fd9e0c08700 1 -- start start 2023-10-06T02:47:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0bee30 0x7fd9dc06a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc06aab0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0c6b10 con 0x7fd9dc0c6200 2023-10-06T02:47:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c9060 con 0x7fd9dc0bee30 2023-10-06T02:47:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9db7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd9dc0c5e30 0x7fd9dc0ab4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57382/0 (socket says 172.21.15.33:57382) 2023-10-06T02:47:02.572 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9db7fe700 1 -- 172.21.15.33:0/1161912832 learned_addr learned my addr 172.21.15.33:0/1161912832 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9daffd700 1 --2- 172.21.15.33:0/1161912832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0bee30 0x7fd9dc06a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701163881 0 0) 0x7fd9dc06aab0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c4003570 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793289463 0 0) 0x7fd9dc0c6b10 con 0x7fd9dc0c6200 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9dc06aab0 con 0x7fd9dc0c6200 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9daffd700 1 -- 172.21.15.33:0/1161912832 >> v1:172.21.15.167:6789/0 conn(0x7fd9dc0c6200 legacy=0x7fd9dc067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9daffd700 1 -- 172.21.15.33:0/1161912832 >> v1:172.21.15.33:6789/0 conn(0x7fd9dc0c5e30 legacy=0x7fd9dc0ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290880187 0 0) 0x7fd9c4003570 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9daffd700 1 -- 172.21.15.33:0/1161912832 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0c8a10 con 0x7fd9dc0bee30 2023-10-06T02:47:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9daffd700 1 --2- 172.21.15.33:0/1161912832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0bee30 0x7fd9dc06a570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009e90 tx=0x7fd9cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8750d0d7783cfb23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00e070 con 0x7fd9dc0bee30 2023-10-06T02:47:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00b870 con 0x7fd9dc0bee30 2023-10-06T02:47:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9da7fc700 1 -- 172.21.15.33:0/1161912832 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0001470 con 0x7fd9dc0bee30 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- 172.21.15.33:0/1161912832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0bee30 msgr2=0x7fd9dc06a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/1161912832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0bee30 0x7fd9dc06a570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009e90 tx=0x7fd9cc009780 comp rx=0 tx=0).stop 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- 172.21.15.33:0/1161912832 shutdown_connections 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/1161912832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0bee30 0x7fd9dc06a570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- 172.21.15.33:0/1161912832 >> 172.21.15.33:0/1161912832 conn(0x7fd9dc0cc2c0 msgr2=0x7fd9dc0c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- 172.21.15.33:0/1161912832 shutdown_connections 2023-10-06T02:47:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.568+0000 7fd9e0c08700 1 -- 172.21.15.33:0/1161912832 wait complete. 2023-10-06T02:47:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 Processor -- start 2023-10-06T02:47:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 -- start start 2023-10-06T02:47:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9dc0bee30 0x7fd9dc0c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 0x7fd9dc1702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0c6200 0x7fd9dc160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c3710 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c46e0 con 0x7fd9dc0bee30 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c3920 con 0x7fd9dc0c6200 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 0x7fd9dc1702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 0x7fd9dc1702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50510/0 (socket says 172.21.15.33:50510) 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 -- 172.21.15.33:0/445450397 learned_addr learned my addr 172.21.15.33:0/445450397 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9db7fe700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9dc0bee30 0x7fd9dc0c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9dbfff700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0c6200 0x7fd9dc160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 -- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0c6200 msgr2=0x7fd9dc160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0c6200 0x7fd9dc160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 -- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9dc0bee30 msgr2=0x7fd9dc0c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9dc0bee30 0x7fd9dc0c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 -- 172.21.15.33:0/445450397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc009a20 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9daffd700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 0x7fd9dc1702b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc06c9a0 tx=0x7fd9cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc012c90 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc012e10 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc0c3c00 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.572+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc01e6a0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.576+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc0c41c0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.576+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc071040 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.576+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd9cc01ead0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.576+0000 7fd9c37fe700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9c8060550 0x7fd9c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.576+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fd9cc07eb40 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.583 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.576+0000 7fd9db7fe700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9c8060550 0x7fd9c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.584 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.580+0000 7fd9db7fe700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9c8060550 0x7fd9c8062a00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00011e0 tx=0x7fd9d0000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:02.587 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.584+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd9cc04f0f0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.756+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd9dc0c8cf0 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.764+0000 7fd9c37fe700 1 -- 172.21.15.33:0/445450397 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9cc052710 con 0x7fd9dc0c5e30 2023-10-06T02:47:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9c8060550 msgr2=0x7fd9c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9c8060550 0x7fd9c8062a00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00011e0 tx=0x7fd9d0000bc0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 msgr2=0x7fd9dc1702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 0x7fd9dc1702b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc06c9a0 tx=0x7fd9cc017000 comp rx=0 tx=0).stop 2023-10-06T02:47:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 shutdown_connections 2023-10-06T02:47:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9c8060550 0x7fd9c8062a00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9dc0c6200 0x7fd9dc160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9dc0c5e30 0x7fd9dc1702b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 --2- 172.21.15.33:0/445450397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9dc0bee30 0x7fd9dc0c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 >> 172.21.15.33:0/445450397 conn(0x7fd9dc0cc2c0 msgr2=0x7fd9dc0c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 shutdown_connections 2023-10-06T02:47:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.768+0000 7fd9e0c08700 1 -- 172.21.15.33:0/445450397 wait complete. 2023-10-06T02:47:02.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_flush.TestFlush.test_flush' 2023-10-06T02:47:02.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 Processor -- start 2023-10-06T02:47:02.884 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 -- start start 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac06e170 0x7fdfac06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 0x7fdfac155070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0c86b0 con 0x7fdfac0c7450 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac068080 con 0x7fdfac154c40 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb2dda700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0c7880 con 0x7fdfac06e170 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb1377700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 0x7fdfac155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb0b76700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac06e170 0x7fdfac06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb1377700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 0x7fdfac155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50514/0 (socket says 172.21.15.33:50514) 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb1377700 1 -- 172.21.15.33:0/1010300135 learned_addr learned my addr 172.21.15.33:0/1010300135 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:02.887 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfabfff700 1 --1- 172.21.15.33:0/1010300135 >> v1:172.21.15.167:6789/0 conn(0x7fdfac0c7450 0x7fdfac067970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.33:56394/0 (socket says 172.21.15.33:56394) 2023-10-06T02:47:02.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1010300135 >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 msgr2=0x7fdfac155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:02.888 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb0b76700 1 --2- 172.21.15.33:0/1010300135 >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 0x7fdfac155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfab7fe700 1 -- 172.21.15.33:0/1010300135 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800976195 0 0) 0x7fdfac0c86b0 con 0x7fdfac0c7450 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1010300135 >> v1:172.21.15.167:6789/0 conn(0x7fdfac0c7450 legacy=0x7fdfac067970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1010300135 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0c7aa0 con 0x7fdfac06e170 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.880+0000 7fdfb1377700 1 --2- 172.21.15.33:0/1010300135 >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 0x7fdfac155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb0b76700 1 --2- 172.21.15.33:0/1010300135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac06e170 0x7fdfac06e540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00bb90 tx=0x7fdf9c00b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7867ee127ce00c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfab7fe700 1 -- 172.21.15.33:0/1010300135 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00da30 con 0x7fdfac06e170 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfab7fe700 1 -- 172.21.15.33:0/1010300135 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c00db90 con 0x7fdfac06e170 2023-10-06T02:47:02.889 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfab7fe700 1 -- 172.21.15.33:0/1010300135 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c017410 con 0x7fdfac06e170 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1010300135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac06e170 msgr2=0x7fdfac06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1010300135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac06e170 0x7fdfac06e540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00bb90 tx=0x7fdf9c00b430 comp rx=0 tx=0).stop 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1010300135 shutdown_connections 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1010300135 >> v2:172.21.15.33:3300/0 conn(0x7fdfac154c40 0x7fdfac155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1010300135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac06e170 0x7fdfac06e540 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1010300135 >> 172.21.15.33:0/1010300135 conn(0x7fdfac06aa50 msgr2=0x7fdfac0cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:02.890 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1010300135 shutdown_connections 2023-10-06T02:47:02.891 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.884+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1010300135 wait complete. 2023-10-06T02:47:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 Processor -- start 2023-10-06T02:47:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 -- start start 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 0x7fdfac1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0c7450 0x7fdfac155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac154c40 0x7fdfac151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac168e90 con 0x7fdfac06e170 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1563b0 con 0x7fdfac0c7450 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac156690 con 0x7fdfac154c40 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 0x7fdfac1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 0x7fdfac1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50516/0 (socket says 172.21.15.33:50516) 2023-10-06T02:47:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1570075355 learned_addr learned my addr 172.21.15.33:0/1570075355 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfabfff700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0c7450 0x7fdfac155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb1377700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac154c40 0x7fdfac151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac154c40 msgr2=0x7fdfac151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac154c40 0x7fdfac151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0c7450 msgr2=0x7fdfac155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0c7450 0x7fdfac155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 -- 172.21.15.33:0/1570075355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4009ae0 con 0x7fdfac06e170 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfabfff700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0c7450 0x7fdfac155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb0b76700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 0x7fdfac1582e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00d6d0 tx=0x7fdf9c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c009070 con 0x7fdfac06e170 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c00a040 con 0x7fdfac06e170 2023-10-06T02:47:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c00b6d0 con 0x7fdfac06e170 2023-10-06T02:47:02.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac156c50 con 0x7fdfac06e170 2023-10-06T02:47:02.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.888+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c020930 con 0x7fdfac06e170 2023-10-06T02:47:02.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.892+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdf9c015070 con 0x7fdfac06e170 2023-10-06T02:47:02.896 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.892+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac0710b0 con 0x7fdfac06e170 2023-10-06T02:47:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.892+0000 7fdfa8ff9700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdf98060550 0x7fdf98062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.892+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fdf9c07dec0 con 0x7fdfac06e170 2023-10-06T02:47:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.892+0000 7fdfabfff700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdf98060550 0x7fdf98062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.892+0000 7fdfabfff700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdf98060550 0x7fdf98062a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfac158cd0 tx=0x7fdfa0007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:02.896+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdf9c04e510 con 0x7fdfac06e170 2023-10-06T02:47:03.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.116+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_flush.TestFlush.test_flush"]} v 0) v1 -- 0x7fdfac06f1a0 con 0x7fdfac06e170 2023-10-06T02:47:03.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.452+0000 7fdfa8ff9700 1 -- 172.21.15.33:0/1570075355 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_flush.TestFlush.test_flush"]}]=0 v109) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdf9c051b30 con 0x7fdfac06e170 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdf98060550 msgr2=0x7fdf98062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdf98060550 0x7fdf98062a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfac158cd0 tx=0x7fdfa0007c90 comp rx=0 tx=0).stop 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 msgr2=0x7fdfac1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 0x7fdfac1582e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00d6d0 tx=0x7fdf9c019000 comp rx=0 tx=0).stop 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 shutdown_connections 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fdf98060550 0x7fdf98062a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdfac154c40 0x7fdfac151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:03.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfac0c7450 0x7fdfac155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:03.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 --2- 172.21.15.33:0/1570075355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfac06e170 0x7fdfac1582e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:03.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 >> 172.21.15.33:0/1570075355 conn(0x7fdfac06aa50 msgr2=0x7fdfac0c1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:03.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 shutdown_connections 2023-10-06T02:47:03.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:03.456+0000 7fdfb2dda700 1 -- 172.21.15.33:0/1570075355 wait complete. 2023-10-06T02:47:03.479 INFO:tasks.cephfs_test_runner:test_flush (tasks.cephfs.test_flush.TestFlush) ... ok 2023-10-06T02:47:03.479 INFO:tasks.cephfs_test_runner: 2023-10-06T02:47:03.479 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T02:47:03.479 INFO:tasks.cephfs_test_runner:Ran 1 test in 52.805s 2023-10-06T02:47:03.479 INFO:tasks.cephfs_test_runner: 2023-10-06T02:47:03.480 INFO:tasks.cephfs_test_runner:OK 2023-10-06T02:47:03.480 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T02:47:03.490 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T02:47:03.521 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T02:47:03.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.522 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2023-10-06T02:47:03.529 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T02:47:03.529 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T02:47:03.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T02:47:03.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T02:47:03.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2023-10-06T02:47:03.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.766 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2023-10-06T02:47:03.774 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T02:47:03.774 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T02:47:03.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.775 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T02:47:03.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.792 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T02:47:03.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T02:47:03.807 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2023-10-06T02:47:03.975 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T02:47:03.984 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T02:47:03.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:47:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.092+0000 7f8b25891700 1 Processor -- start 2023-10-06T02:47:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.092+0000 7f8b25891700 1 -- start start 2023-10-06T02:47:04.100 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b200c9550 0x7f8b200680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200c99d0 con 0x7f8b20160fd0 2023-10-06T02:47:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b20067050 con 0x7f8b20068630 2023-10-06T02:47:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b20067330 con 0x7f8b200c9550 2023-10-06T02:47:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b200c9550 0x7f8b200680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8b200c9550 0x7f8b200680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50528/0 (socket says 172.21.15.33:50528) 2023-10-06T02:47:04.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/2086430883 learned_addr learned my addr 172.21.15.33:0/2086430883 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:04.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/2086430883 >> v1:172.21.15.167:6789/0 conn(0x7f8b20068630 legacy=0x7f8b200c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/2086430883 >> v1:172.21.15.33:6789/0 conn(0x7f8b20160fd0 legacy=0x7f8b201613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1dffb700 1 -- 172.21.15.33:0/2086430883 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122633044 0 0) 0x7f8b200c99d0 con 0x7f8b20160fd0 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/2086430883 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20067610 con 0x7f8b200c9550 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1e7fc700 1 --2- 172.21.15.33:0/2086430883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200680f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8b100099b0 tx=0x7f8b10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f769dec1319b385b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1dffb700 1 -- 172.21.15.33:0/2086430883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000b860 con 0x7f8b200c9550 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1dffb700 1 -- 172.21.15.33:0/2086430883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1000b9c0 con 0x7f8b200c9550 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b1dffb700 1 -- 172.21.15.33:0/2086430883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000bd60 con 0x7f8b200c9550 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- 172.21.15.33:0/2086430883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 msgr2=0x7f8b200680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 --2- 172.21.15.33:0/2086430883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200680f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8b100099b0 tx=0x7f8b10009250 comp rx=0 tx=0).stop 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- 172.21.15.33:0/2086430883 shutdown_connections 2023-10-06T02:47:04.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 --2- 172.21.15.33:0/2086430883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200680f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- 172.21.15.33:0/2086430883 >> 172.21.15.33:0/2086430883 conn(0x7f8b200c83f0 msgr2=0x7f8b200c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- 172.21.15.33:0/2086430883 shutdown_connections 2023-10-06T02:47:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.096+0000 7f8b25891700 1 -- 172.21.15.33:0/2086430883 wait complete. 2023-10-06T02:47:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 Processor -- start 2023-10-06T02:47:04.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 -- start start 2023-10-06T02:47:04.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b20068630 0x7f8b20159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.107 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b20160fd0 0x7f8b20159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200af7c0 con 0x7f8b200c9550 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200c4020 con 0x7f8b20068630 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b25891700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200c4300 con 0x7f8b20160fd0 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50536/0 (socket says 172.21.15.33:50536) 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/16433746 learned_addr learned my addr 172.21.15.33:0/16433746 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1effd700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b20068630 0x7f8b20159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1f7fe700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b20160fd0 0x7f8b20159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b20160fd0 msgr2=0x7f8b20159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b20160fd0 0x7f8b20159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.108 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b20068630 msgr2=0x7f8b20159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b20068630 0x7f8b20159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/16433746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100094f0 con 0x7f8b200c9550 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1effd700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b20068630 0x7f8b20159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1f7fe700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b20160fd0 0x7f8b20159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b1e7fc700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200afc50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8b10006f50 tx=0x7f8b10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1001f070 con 0x7f8b200c9550 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10011d10 con 0x7f8b200c9550 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.100+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10025670 con 0x7f8b200c9550 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.104+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b200c45e0 con 0x7f8b200c9550 2023-10-06T02:47:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.104+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2015a230 con 0x7f8b200c9550 2023-10-06T02:47:04.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.104+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b200af930 con 0x7f8b200c9550 2023-10-06T02:47:04.110 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.104+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8b10023070 con 0x7f8b200c9550 2023-10-06T02:47:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.104+0000 7f8b06ffd700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b0c060580 0x7f8b0c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.104+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f8b1007dee0 con 0x7f8b200c9550 2023-10-06T02:47:04.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.108+0000 7f8b1effd700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b0c060580 0x7f8b0c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.108+0000 7f8b1effd700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b0c060580 0x7f8b0c062a30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b140011e0 tx=0x7f8b14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.112+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8b1004e5b0 con 0x7f8b200c9550 2023-10-06T02:47:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.280+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b200710b0 con 0x7f8b200c9550 2023-10-06T02:47:04.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.280+0000 7f8b06ffd700 1 -- 172.21.15.33:0/16433746 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13984 (secure 0 0 0) 0x7f8b10051b50 con 0x7f8b200c9550 2023-10-06T02:47:04.287 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:04.287 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:47:02.427992+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/3594813995":"2023-10-07T02:46:59.054601+0000","172.21.15.33:6834/3594813995":"2023-10-07T02:46:59.054601+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:47:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b0c060580 msgr2=0x7f8b0c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b0c060580 0x7f8b0c062a30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b140011e0 tx=0x7f8b14009040 comp rx=0 tx=0).stop 2023-10-06T02:47:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 msgr2=0x7f8b200afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200afc50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8b10006f50 tx=0x7f8b10017000 comp rx=0 tx=0).stop 2023-10-06T02:47:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 shutdown_connections 2023-10-06T02:47:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f8b0c060580 0x7f8b0c062a30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b20160fd0 0x7f8b20159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200c9550 0x7f8b200afc50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 --2- 172.21.15.33:0/16433746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b20068630 0x7f8b20159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 >> 172.21.15.33:0/16433746 conn(0x7f8b200c83f0 msgr2=0x7f8b20069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:04.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 shutdown_connections 2023-10-06T02:47:04.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.284+0000 7f8b25891700 1 -- 172.21.15.33:0/16433746 wait complete. 2023-10-06T02:47:04.301 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T02:47:04.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:47:04.410 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.404+0000 7f19e9a36700 1 Processor -- start 2023-10-06T02:47:04.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.404+0000 7f19e9a36700 1 -- start start 2023-10-06T02:47:04.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19e40adb60 0x7f19e40c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e415ace0 0x7f19e4069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e4157eb0 con 0x7f19e4067970 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e4069820 con 0x7f19e40adb60 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4067ea0 con 0x7f19e415ace0 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19e40adb60 0x7f19e40c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f19e40adb60 0x7f19e40c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50546/0 (socket says 172.21.15.33:50546) 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 -- 172.21.15.33:0/30833383 learned_addr learned my addr 172.21.15.33:0/30833383 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9235700 1 --2- 172.21.15.33:0/30833383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e415ace0 0x7f19e4069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 -- 172.21.15.33:0/30833383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e415ace0 msgr2=0x7f19e4069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 --2- 172.21.15.33:0/30833383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e415ace0 0x7f19e4069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 -- 172.21.15.33:0/30833383 >> v1:172.21.15.167:6789/0 conn(0x7f19e4067970 legacy=0x7f19e415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 -- 172.21.15.33:0/30833383 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4068180 con 0x7f19e40adb60 2023-10-06T02:47:04.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e8a34700 1 --2- 172.21.15.33:0/30833383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c6820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5fff6a3d73c47b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e37fe700 1 -- 172.21.15.33:0/30833383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800b860 con 0x7f19e40adb60 2023-10-06T02:47:04.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e37fe700 1 -- 172.21.15.33:0/30833383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d800b9c0 con 0x7f19e40adb60 2023-10-06T02:47:04.416 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e37fe700 1 -- 172.21.15.33:0/30833383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800bd60 con 0x7f19e40adb60 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- 172.21.15.33:0/30833383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 msgr2=0x7f19e40c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 --2- 172.21.15.33:0/30833383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c6820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8009250 comp rx=0 tx=0).stop 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- 172.21.15.33:0/30833383 shutdown_connections 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 --2- 172.21.15.33:0/30833383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e415ace0 0x7f19e4069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 --2- 172.21.15.33:0/30833383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c6820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- 172.21.15.33:0/30833383 >> 172.21.15.33:0/30833383 conn(0x7f19e40c8240 msgr2=0x7f19e40c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.408+0000 7f19e9a36700 1 -- 172.21.15.33:0/30833383 shutdown_connections 2023-10-06T02:47:04.417 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.412+0000 7f19e9a36700 1 -- 172.21.15.33:0/30833383 wait complete. 2023-10-06T02:47:04.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.412+0000 7f19e9a36700 1 Processor -- start 2023-10-06T02:47:04.418 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.412+0000 7f19e9a36700 1 -- start start 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e4067970 0x7f19e41514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e415ace0 0x7f19e4166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e416a880 con 0x7f19e40adb60 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4167720 con 0x7f19e415ace0 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4167a00 con 0x7f19e4067970 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e8a34700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e4067970 0x7f19e41514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50548/0 (socket says 172.21.15.33:50548) 2023-10-06T02:47:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 -- 172.21.15.33:0/3832973160 learned_addr learned my addr 172.21.15.33:0/3832973160 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9235700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e415ace0 0x7f19e4166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 -- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e4067970 msgr2=0x7f19e41514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e4067970 0x7f19e41514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 -- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e415ace0 msgr2=0x7f19e4166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e415ace0 0x7f19e4166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 -- 172.21.15.33:0/3832973160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80094f0 con 0x7f19e40adb60 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e8a34700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e4067970 0x7f19e41514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e3fff700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c38d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c590 tx=0x7f19d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0011590 con 0x7f19e40adb60 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d0011710 con 0x7f19e40adb60 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4167ce0 con 0x7f19e40adb60 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0011ad0 con 0x7f19e40adb60 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e4167420 con 0x7f19e40adb60 2023-10-06T02:47:04.424 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.416+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e416aa30 con 0x7f19e40adb60 2023-10-06T02:47:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.420+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f19d001b3b0 con 0x7f19e40adb60 2023-10-06T02:47:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.420+0000 7f19e0ff9700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19c8060550 0x7f19c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.420+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f19d007ba70 con 0x7f19e40adb60 2023-10-06T02:47:04.425 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.420+0000 7f19e8a34700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19c8060550 0x7f19c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.420+0000 7f19e8a34700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19c8060550 0x7f19c8062a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.428+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19e416aa30 con 0x7f19e40adb60 2023-10-06T02:47:04.589 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.584+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19e4071040 con 0x7f19c8060550 2023-10-06T02:47:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e0ff9700 1 -- 172.21.15.33:0/3832973160 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71324 (secure 0 0 0) 0x7f19e4071040 con 0x7f19c8060550 2023-10-06T02:47:04.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19c8060550 msgr2=0x7f19c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19c8060550 0x7f19c8062a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099b0 tx=0x7f19d8017000 comp rx=0 tx=0).stop 2023-10-06T02:47:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 msgr2=0x7f19e40c38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c38d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c590 tx=0x7f19d000c8a0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 shutdown_connections 2023-10-06T02:47:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e415ace0 0x7f19e4166ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e40adb60 0x7f19e40c38d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f19c8060550 0x7f19c8062a00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 --2- 172.21.15.33:0/3832973160 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f19e4067970 0x7f19e41514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 >> 172.21.15.33:0/3832973160 conn(0x7f19e40c8240 msgr2=0x7f19e41597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 shutdown_connections 2023-10-06T02:47:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.592+0000 7f19e9a36700 1 -- 172.21.15.33:0/3832973160 wait complete. 2023-10-06T02:47:04.600 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:47:04.614 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-10-06T02:47:02.594715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391760,"kb_used_data":5264,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752582960,"statfs":{"total":773094113280,"available":770644951040,"internally_reserved":0,"allocated":5390336,"data_stored":3055840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22489,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-13,"num_read_kb":-9,"num_write":-107,"num_write_kb":-59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002486"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:29.321321+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:46:29.321321+0000","last_peered":"2023-10-06T02:46:29.321321+0000","last_clean":"2023-10-06T02:46:29.321321+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:46:29.321321+0000","last_undegraded":"2023-10-06T02:46:29.321321+0000","last_fullsized":"2023-10-06T02:46:29.321321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.317342+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:46:28.317342+0000","last_peered":"2023-10-06T02:46:28.317342+0000","last_clean":"2023-10-06T02:46:28.317342+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:46:28.317342+0000","last_undegraded":"2023-10-06T02:46:28.317342+0000","last_fullsized":"2023-10-06T02:46:28.317342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.018551+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:46:30.018551+0000","last_peered":"2023-10-06T02:46:30.018551+0000","last_clean":"2023-10-06T02:46:30.018551+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:46:30.018551+0000","last_undegraded":"2023-10-06T02:46:30.018551+0000","last_fullsized":"2023-10-06T02:46:30.018551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.316250+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:46:28.316250+0000","last_peered":"2023-10-06T02:46:28.316250+0000","last_clean":"2023-10-06T02:46:28.316250+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:46:28.316250+0000","last_undegraded":"2023-10-06T02:46:28.316250+0000","last_fullsized":"2023-10-06T02:46:28.316250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.015469+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:46:30.015469+0000","last_peered":"2023-10-06T02:46:30.015469+0000","last_clean":"2023-10-06T02:46:30.015469+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:46:30.015469+0000","last_undegraded":"2023-10-06T02:46:30.015469+0000","last_fullsized":"2023-10-06T02:46:30.015469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:29.314831+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:46:29.314831+0000","last_peered":"2023-10-06T02:46:29.314831+0000","last_clean":"2023-10-06T02:46:29.314831+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:46:29.314831+0000","last_undegraded":"2023-10-06T02:46:29.314831+0000","last_fullsized":"2023-10-06T02:46:29.314831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.017264+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:46:30.017264+0000","last_peered":"2023-10-06T02:46:30.017264+0000","last_clean":"2023-10-06T02:46:30.017264+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:46:30.017264+0000","last_undegraded":"2023-10-06T02:46:30.017264+0000","last_fullsized":"2023-10-06T02:46:30.017264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.015851+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:46:30.015851+0000","last_peered":"2023-10-06T02:46:30.015851+0000","last_clean":"2023-10-06T02:46:30.015851+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:46:30.015851+0000","last_undegraded":"2023-10-06T02:46:30.015851+0000","last_fullsized":"2023-10-06T02:46:30.015851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.017506+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:46:30.017506+0000","last_peered":"2023-10-06T02:46:30.017506+0000","last_clean":"2023-10-06T02:46:30.017506+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:46:30.017506+0000","last_undegraded":"2023-10-06T02:46:30.017506+0000","last_fullsized":"2023-10-06T02:46:30.017506+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":824245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2869999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.0409999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301904,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069936,"statfs":{"total":96636764160,"available":96327614464,"internally_reserved":0,"allocated":532480,"data_stored":234633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.093},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.159}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.131}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2210000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.2649999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":227115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0329999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.214}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.2909999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.002}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":540672,"data_stored":248703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.014}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.097}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.359}]},{"osd":7,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.98799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":227655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1779999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.117}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.21},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.0589999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":248725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.91500000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.014},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.339}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0760000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.054},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.8089999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":227317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.147}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.93999999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":817447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.097},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":1.069}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:47:04.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:47:04.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 Processor -- start 2023-10-06T02:47:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 -- start start 2023-10-06T02:47:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 0x7f3fc80680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fc8069580 0x7f3fc8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8156fe0 con 0x7f3fc8156100 2023-10-06T02:47:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc806a970 con 0x7f3fc8069580 2023-10-06T02:47:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fd0563700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc806a380 con 0x7f3fc8155d30 2023-10-06T02:47:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 0x7f3fc80680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 0x7f3fc80680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:45208/0 (socket says 172.21.15.33:45208) 2023-10-06T02:47:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fceb00700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3fc8069580 0x7f3fc8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1756960878 learned_addr learned my addr 172.21.15.33:0/1756960878 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1756960878 >> v2:172.21.15.167:3300/0 conn(0x7f3fc8069580 msgr2=0x7f3fc8159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 --2- 172.21.15.33:0/1756960878 >> v2:172.21.15.167:3300/0 conn(0x7f3fc8069580 0x7f3fc8159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fcd2fd700 1 -- 172.21.15.33:0/1756960878 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101256467 0 0) 0x7f3fc8156fe0 con 0x7f3fc8156100 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1756960878 >> v1:172.21.15.167:6789/0 conn(0x7f3fc8156100 legacy=0x7f3fc8068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1756960878 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc81572c0 con 0x7f3fc8155d30 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.720+0000 7f3fce2ff700 1 --2- 172.21.15.33:0/1756960878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 0x7f3fc80680c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400bc30 tx=0x7f3fc400b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a59d303751a58a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fcd2fd700 1 -- 172.21.15.33:0/1756960878 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4005c10 con 0x7f3fc8155d30 2023-10-06T02:47:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fcd2fd700 1 -- 172.21.15.33:0/1756960878 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc4005d70 con 0x7f3fc8155d30 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fcd2fd700 1 -- 172.21.15.33:0/1756960878 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400dbc0 con 0x7f3fc8155d30 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 -- 172.21.15.33:0/1756960878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 msgr2=0x7f3fc80680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1756960878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 0x7f3fc80680c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400bc30 tx=0x7f3fc400b4c0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 -- 172.21.15.33:0/1756960878 shutdown_connections 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1756960878 >> v2:172.21.15.167:3300/0 conn(0x7f3fc8069580 0x7f3fc8159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1756960878 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8155d30 0x7f3fc80680c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 -- 172.21.15.33:0/1756960878 >> 172.21.15.33:0/1756960878 conn(0x7f3fc80af920 msgr2=0x7f3fc80afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 -- 172.21.15.33:0/1756960878 shutdown_connections 2023-10-06T02:47:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 -- 172.21.15.33:0/1756960878 wait complete. 2023-10-06T02:47:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 Processor -- start 2023-10-06T02:47:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.724+0000 7f3fd0563700 1 -- start start 2023-10-06T02:47:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8069580 0x7f3fc814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc8155d30 0x7f3fc8164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc8156100 0x7f3fc80cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc814f200 con 0x7f3fc8156100 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8164e70 con 0x7f3fc8155d30 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80cc8b0 con 0x7f3fc8069580 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fceb00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc8156100 0x7f3fc80cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fceb00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc8156100 0x7f3fc80cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50550/0 (socket says 172.21.15.33:50550) 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8069580 0x7f3fc814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fceb00700 1 -- 172.21.15.33:0/1608906015 learned_addr learned my addr 172.21.15.33:0/1608906015 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fcdafe700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc8155d30 0x7f3fc8164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc8155d30 msgr2=0x7f3fc8164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.734 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc8155d30 0x7f3fc8164930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1608906015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc8156100 msgr2=0x7f3fc80cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc8156100 0x7f3fc80cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 -- 172.21.15.33:0/1608906015 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc400b720 con 0x7f3fc8069580 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fcdafe700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc8155d30 0x7f3fc8164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fce2ff700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8069580 0x7f3fc814ecc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4017000 tx=0x7f3fc401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400dba0 con 0x7f3fc8069580 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc401d070 con 0x7f3fc8069580 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc814f6d0 con 0x7f3fc8069580 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4024670 con 0x7f3fc8069580 2023-10-06T02:47:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc8139760 con 0x7f3fc8069580 2023-10-06T02:47:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.728+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc81395c0 con 0x7f3fc8069580 2023-10-06T02:47:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.732+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3fc40249e0 con 0x7f3fc8069580 2023-10-06T02:47:04.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.732+0000 7f3fba7fc700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fac060550 0x7f3fac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:04.737 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.732+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f3fc4014030 con 0x7f3fc8069580 2023-10-06T02:47:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.732+0000 7f3fcdafe700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fac060550 0x7f3fac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.732+0000 7f3fcdafe700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fac060550 0x7f3fac062a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3fc81534e0 tx=0x7f3fbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.744+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3fc4051e60 con 0x7f3fc8069580 2023-10-06T02:47:04.911 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.904+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fc814f6d0 con 0x7f3fac060550 2023-10-06T02:47:04.918 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fba7fc700 1 -- 172.21.15.33:0/1608906015 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71324 (secure 0 0 0) 0x7f3fc814f6d0 con 0x7f3fac060550 2023-10-06T02:47:04.919 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fac060550 msgr2=0x7f3fac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fac060550 0x7f3fac062a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3fc81534e0 tx=0x7f3fbc009040 comp rx=0 tx=0).stop 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8069580 msgr2=0x7f3fc814ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8069580 0x7f3fc814ecc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4017000 tx=0x7f3fc401a000 comp rx=0 tx=0).stop 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 shutdown_connections 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc8156100 0x7f3fc80cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fc8155d30 0x7f3fc8164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f3fac060550 0x7f3fac062a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 --2- 172.21.15.33:0/1608906015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fc8069580 0x7f3fc814ecc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 >> 172.21.15.33:0/1608906015 conn(0x7f3fc80af920 msgr2=0x7f3fc80afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 shutdown_connections 2023-10-06T02:47:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:04.916+0000 7f3fd0563700 1 -- 172.21.15.33:0/1608906015 wait complete. 2023-10-06T02:47:04.922 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:47:04.934 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-10-06T02:47:04.595613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391760,"kb_used_data":5264,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752582960,"statfs":{"total":773094113280,"available":770644951040,"internally_reserved":0,"allocated":5390336,"data_stored":3055840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22489,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-13,"num_read_kb":-9,"num_write":-107,"num_write_kb":-59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002424"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:29.321321+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:46:29.321321+0000","last_peered":"2023-10-06T02:46:29.321321+0000","last_clean":"2023-10-06T02:46:29.321321+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:46:29.321321+0000","last_undegraded":"2023-10-06T02:46:29.321321+0000","last_fullsized":"2023-10-06T02:46:29.321321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.317342+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:46:28.317342+0000","last_peered":"2023-10-06T02:46:28.317342+0000","last_clean":"2023-10-06T02:46:28.317342+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:46:28.317342+0000","last_undegraded":"2023-10-06T02:46:28.317342+0000","last_fullsized":"2023-10-06T02:46:28.317342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.018551+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:46:30.018551+0000","last_peered":"2023-10-06T02:46:30.018551+0000","last_clean":"2023-10-06T02:46:30.018551+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:46:30.018551+0000","last_undegraded":"2023-10-06T02:46:30.018551+0000","last_fullsized":"2023-10-06T02:46:30.018551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.316250+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:46:28.316250+0000","last_peered":"2023-10-06T02:46:28.316250+0000","last_clean":"2023-10-06T02:46:28.316250+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:46:28.316250+0000","last_undegraded":"2023-10-06T02:46:28.316250+0000","last_fullsized":"2023-10-06T02:46:28.316250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.015469+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:46:30.015469+0000","last_peered":"2023-10-06T02:46:30.015469+0000","last_clean":"2023-10-06T02:46:30.015469+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:46:30.015469+0000","last_undegraded":"2023-10-06T02:46:30.015469+0000","last_fullsized":"2023-10-06T02:46:30.015469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:29.314831+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:46:29.314831+0000","last_peered":"2023-10-06T02:46:29.314831+0000","last_clean":"2023-10-06T02:46:29.314831+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:46:29.314831+0000","last_undegraded":"2023-10-06T02:46:29.314831+0000","last_fullsized":"2023-10-06T02:46:29.314831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.017264+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:46:30.017264+0000","last_peered":"2023-10-06T02:46:30.017264+0000","last_clean":"2023-10-06T02:46:30.017264+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:46:30.017264+0000","last_undegraded":"2023-10-06T02:46:30.017264+0000","last_fullsized":"2023-10-06T02:46:30.017264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.015851+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:46:30.015851+0000","last_peered":"2023-10-06T02:46:30.015851+0000","last_clean":"2023-10-06T02:46:30.015851+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:46:30.015851+0000","last_undegraded":"2023-10-06T02:46:30.015851+0000","last_fullsized":"2023-10-06T02:46:30.015851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.017506+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:46:30.017506+0000","last_peered":"2023-10-06T02:46:30.017506+0000","last_clean":"2023-10-06T02:46:30.017506+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:46:30.017506+0000","last_undegraded":"2023-10-06T02:46:30.017506+0000","last_fullsized":"2023-10-06T02:46:30.017506+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":824245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2869999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.0409999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301904,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069936,"statfs":{"total":96636764160,"available":96327614464,"internally_reserved":0,"allocated":532480,"data_stored":234633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.093},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.159}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.131}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2210000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.2649999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":227115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0329999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.214}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.2909999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.002}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":540672,"data_stored":248703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.014}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.097}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.359}]},{"osd":7,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.98799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":227655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1779999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.117}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.21},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.0589999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":248725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.91500000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.014},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.339}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0760000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.054},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.8089999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":227317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.147}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.93999999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":817447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.097},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":1.069}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:47:04.937 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T02:47:04.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:47:05.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 Processor -- start 2023-10-06T02:47:05.049 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 -- start start 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8480ade00 0x7fd8480c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd8480c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84806add0 con 0x7fd8480c5940 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480c80c0 con 0x7fd8480ade00 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84f1f8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480c7c40 con 0x7fd848066fb0 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8480ade00 0x7fd8480c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd8480ade00 0x7fd8480c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50566/0 (socket says 172.21.15.33:50566) 2023-10-06T02:47:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 -- 172.21.15.33:0/3500477264 learned_addr learned my addr 172.21.15.33:0/3500477264 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e9f7700 1 --2- 172.21.15.33:0/3500477264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd8480c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 -- 172.21.15.33:0/3500477264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 msgr2=0x7fd8480c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:05.052 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 --2- 172.21.15.33:0/3500477264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd8480c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 -- 172.21.15.33:0/3500477264 >> v1:172.21.15.167:6789/0 conn(0x7fd8480c5940 legacy=0x7fd8480c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 -- 172.21.15.33:0/3500477264 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480ae1d0 con 0x7fd8480ade00 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84e1f6700 1 --2- 172.21.15.33:0/3500477264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480ade00 0x7fd8480c71c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd8380099e0 tx=0x7fd838009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4a0b8e5d54a2ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84d1f4700 1 -- 172.21.15.33:0/3500477264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83800b820 con 0x7fd8480ade00 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84d1f4700 1 -- 172.21.15.33:0/3500477264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83800b980 con 0x7fd8480ade00 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.044+0000 7fd84d1f4700 1 -- 172.21.15.33:0/3500477264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83800bd20 con 0x7fd8480ade00 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 -- 172.21.15.33:0/3500477264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480ade00 msgr2=0x7fd8480c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/3500477264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480ade00 0x7fd8480c71c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd8380099e0 tx=0x7fd838009250 comp rx=0 tx=0).stop 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 -- 172.21.15.33:0/3500477264 shutdown_connections 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/3500477264 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd8480c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/3500477264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480ade00 0x7fd8480c71c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 -- 172.21.15.33:0/3500477264 >> 172.21.15.33:0/3500477264 conn(0x7fd8480c8240 msgr2=0x7fd8480c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 -- 172.21.15.33:0/3500477264 shutdown_connections 2023-10-06T02:47:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 -- 172.21.15.33:0/3500477264 wait complete. 2023-10-06T02:47:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 Processor -- start 2023-10-06T02:47:05.055 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.048+0000 7fd84f1f8700 1 -- start start 2023-10-06T02:47:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd84814b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8480ade00 0x7fd8481495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 0x7fd8481566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480c4460 con 0x7fd8480c5940 2023-10-06T02:47:05.058 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480c3850 con 0x7fd8480ade00 2023-10-06T02:47:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480c3b30 con 0x7fd848066fb0 2023-10-06T02:47:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84e9f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 0x7fd8481566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84e9f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 0x7fd8481566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50572/0 (socket says 172.21.15.33:50572) 2023-10-06T02:47:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84e9f7700 1 -- 172.21.15.33:0/2657975270 learned_addr learned my addr 172.21.15.33:0/2657975270 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84e1f6700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd84814b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8480ade00 0x7fd8481495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 -- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 msgr2=0x7fd84814b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd84814b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 -- 172.21.15.33:0/2657975270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 msgr2=0x7fd8481566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 0x7fd8481566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 -- 172.21.15.33:0/2657975270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380094f0 con 0x7fd8480ade00 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84e9f7700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 0x7fd8481566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84d9f5700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8480ade00 0x7fd8481495f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd844001eb0 tx=0x7fd844007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400ce10 con 0x7fd8480ade00 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844011590 con 0x7fd8480ade00 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844011950 con 0x7fd8480ade00 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8480c3e10 con 0x7fd8480ade00 2023-10-06T02:47:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.052+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84814b980 con 0x7fd8480ade00 2023-10-06T02:47:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.056+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8480c7aa0 con 0x7fd8480ade00 2023-10-06T02:47:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.056+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd844011d00 con 0x7fd8480ade00 2023-10-06T02:47:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.056+0000 7fd83e7fc700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd834060550 0x7fd834062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.056+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fd84407ba10 con 0x7fd8480ade00 2023-10-06T02:47:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.056+0000 7fd84e1f6700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd834060550 0x7fd834062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.060+0000 7fd84e1f6700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd834060550 0x7fd834062a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd8380099e0 tx=0x7fd838017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.068+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd8480c7aa0 con 0x7fd8480ade00 2023-10-06T02:47:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.224+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8480c7aa0 con 0x7fd834060550 2023-10-06T02:47:05.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.232+0000 7fd83e7fc700 1 -- 172.21.15.33:0/2657975270 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71324 (secure 0 0 0) 0x7fd8480c7aa0 con 0x7fd834060550 2023-10-06T02:47:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:05.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd834060550 msgr2=0x7fd834062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd834060550 0x7fd834062a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd8380099e0 tx=0x7fd838017000 comp rx=0 tx=0).stop 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8480ade00 msgr2=0x7fd8481495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8480ade00 0x7fd8481495f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd844001eb0 tx=0x7fd844007ec0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 shutdown_connections 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8480c5940 0x7fd8481566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8480ade00 0x7fd8481495f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd834060550 0x7fd834062a00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 --2- 172.21.15.33:0/2657975270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd848066fb0 0x7fd84814b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 >> 172.21.15.33:0/2657975270 conn(0x7fd8480c8240 msgr2=0x7fd848069310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:05.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 shutdown_connections 2023-10-06T02:47:05.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.236+0000 7fd84f1f8700 1 -- 172.21.15.33:0/2657975270 wait complete. 2023-10-06T02:47:05.244 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:47:05.257 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-10-06T02:47:04.595613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391760,"kb_used_data":5264,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752582960,"statfs":{"total":773094113280,"available":770644951040,"internally_reserved":0,"allocated":5390336,"data_stored":3055840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22489,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-13,"num_read_kb":-9,"num_write":-107,"num_write_kb":-59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002424"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:29.321321+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:46:29.321321+0000","last_peered":"2023-10-06T02:46:29.321321+0000","last_clean":"2023-10-06T02:46:29.321321+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:46:29.321321+0000","last_undegraded":"2023-10-06T02:46:29.321321+0000","last_fullsized":"2023-10-06T02:46:29.321321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.317342+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:46:28.317342+0000","last_peered":"2023-10-06T02:46:28.317342+0000","last_clean":"2023-10-06T02:46:28.317342+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:46:28.317342+0000","last_undegraded":"2023-10-06T02:46:28.317342+0000","last_fullsized":"2023-10-06T02:46:28.317342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.018551+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:46:30.018551+0000","last_peered":"2023-10-06T02:46:30.018551+0000","last_clean":"2023-10-06T02:46:30.018551+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:46:30.018551+0000","last_undegraded":"2023-10-06T02:46:30.018551+0000","last_fullsized":"2023-10-06T02:46:30.018551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T02:46:28.316250+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:46:28.316250+0000","last_peered":"2023-10-06T02:46:28.316250+0000","last_clean":"2023-10-06T02:46:28.316250+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:46:28.316250+0000","last_undegraded":"2023-10-06T02:46:28.316250+0000","last_fullsized":"2023-10-06T02:46:28.316250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:12.826817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.015469+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:46:30.015469+0000","last_peered":"2023-10-06T02:46:30.015469+0000","last_clean":"2023-10-06T02:46:30.015469+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:46:30.015469+0000","last_undegraded":"2023-10-06T02:46:30.015469+0000","last_fullsized":"2023-10-06T02:46:30.015469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:29.314831+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:46:29.314831+0000","last_peered":"2023-10-06T02:46:29.314831+0000","last_clean":"2023-10-06T02:46:29.314831+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:46:29.314831+0000","last_undegraded":"2023-10-06T02:46:29.314831+0000","last_fullsized":"2023-10-06T02:46:29.314831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.017264+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:46:30.017264+0000","last_peered":"2023-10-06T02:46:30.017264+0000","last_clean":"2023-10-06T02:46:30.017264+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:46:30.017264+0000","last_undegraded":"2023-10-06T02:46:30.017264+0000","last_fullsized":"2023-10-06T02:46:30.017264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.015851+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:46:30.015851+0000","last_peered":"2023-10-06T02:46:30.015851+0000","last_clean":"2023-10-06T02:46:30.015851+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:46:30.015851+0000","last_undegraded":"2023-10-06T02:46:30.015851+0000","last_fullsized":"2023-10-06T02:46:30.015851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:48.137674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T02:46:30.017506+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:46:30.017506+0000","last_peered":"2023-10-06T02:46:30.017506+0000","last_clean":"2023-10-06T02:46:30.017506+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:46:30.017506+0000","last_undegraded":"2023-10-06T02:46:30.017506+0000","last_fullsized":"2023-10-06T02:46:30.017506+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":824245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2869999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.0409999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301904,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069936,"statfs":{"total":96636764160,"available":96327614464,"internally_reserved":0,"allocated":532480,"data_stored":234633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.093},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.159}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.131}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.2210000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.2649999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297788,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074052,"statfs":{"total":96636764160,"available":96331829248,"internally_reserved":0,"allocated":512000,"data_stored":227115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0329999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.214}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.2909999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.002}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":540672,"data_stored":248703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.014}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.097}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.359}]},{"osd":7,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.98799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":520192,"data_stored":227655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.747}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1779999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.117}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.21},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.0589999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":248725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.91500000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.014},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.339}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0760000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.054},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.8089999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":227317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.147}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.224},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.93999999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":817447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.097},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":1.069}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:47:05.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T02:47:05.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96ce377700 1 Processor -- start 2023-10-06T02:47:05.364 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96ce377700 1 -- start start 2023-10-06T02:47:05.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96ce377700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8158d00 0x7f96c8068450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.365 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96ce377700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c80cdf80 con 0x7f96c80cf420 2023-10-06T02:47:05.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96ce377700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c8068de0 con 0x7f96c8068990 2023-10-06T02:47:05.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96ce377700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c815da10 con 0x7f96c8158d00 2023-10-06T02:47:05.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cdb76700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f96c80cf420 0x7f96c81590d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45080/0 (socket says 172.21.15.33:45080) 2023-10-06T02:47:05.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cdb76700 1 -- 172.21.15.33:0/1830107066 learned_addr learned my addr 172.21.15.33:0/1830107066 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:05.366 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cd375700 1 --2- 172.21.15.33:0/1830107066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8158d00 0x7f96c8068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96bffff700 1 -- 172.21.15.33:0/1830107066 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500623395 0 0) 0x7f96c80cdf80 con 0x7f96c80cf420 2023-10-06T02:47:05.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96bffff700 1 -- 172.21.15.33:0/1830107066 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b4003570 con 0x7f96c80cf420 2023-10-06T02:47:05.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cd375700 1 -- 172.21.15.33:0/1830107066 >> v1:172.21.15.167:6789/0 conn(0x7f96c8068990 legacy=0x7f96c815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.367 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cd375700 1 -- 172.21.15.33:0/1830107066 >> v1:172.21.15.33:6789/0 conn(0x7f96c80cf420 legacy=0x7f96c81590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cd375700 1 -- 172.21.15.33:0/1830107066 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c815d730 con 0x7f96c8158d00 2023-10-06T02:47:05.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96cd375700 1 --2- 172.21.15.33:0/1830107066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8158d00 0x7f96c8068450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009910 tx=0x7f96c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eee5bb409d55810 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:05.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96bffff700 1 -- 172.21.15.33:0/1830107066 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c400b680 con 0x7f96c8158d00 2023-10-06T02:47:05.368 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96bffff700 1 -- 172.21.15.33:0/1830107066 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c400b7e0 con 0x7f96c8158d00 2023-10-06T02:47:05.369 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.360+0000 7f96bffff700 1 -- 172.21.15.33:0/1830107066 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c400bba0 con 0x7f96c8158d00 2023-10-06T02:47:05.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 -- 172.21.15.33:0/1830107066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8158d00 msgr2=0x7f96c8068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 --2- 172.21.15.33:0/1830107066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8158d00 0x7f96c8068450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009910 tx=0x7f96c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 -- 172.21.15.33:0/1830107066 shutdown_connections 2023-10-06T02:47:05.370 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 --2- 172.21.15.33:0/1830107066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8158d00 0x7f96c8068450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 -- 172.21.15.33:0/1830107066 >> 172.21.15.33:0/1830107066 conn(0x7f96c8067a80 msgr2=0x7f96c80cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:05.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 -- 172.21.15.33:0/1830107066 shutdown_connections 2023-10-06T02:47:05.371 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 -- 172.21.15.33:0/1830107066 wait complete. 2023-10-06T02:47:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 Processor -- start 2023-10-06T02:47:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.364+0000 7f96ce377700 1 -- start start 2023-10-06T02:47:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8068990 0x7f96c8147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 0x7f96c80b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96c8158d00 0x7f96c80c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80c8750 con 0x7f96c80cf420 2023-10-06T02:47:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8148110 con 0x7f96c8158d00 2023-10-06T02:47:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80c82c0 con 0x7f96c8068990 2023-10-06T02:47:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 0x7f96c80b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 0x7f96c80b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50582/0 (socket says 172.21.15.33:50582) 2023-10-06T02:47:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96cd375700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8068990 0x7f96c8147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 -- 172.21.15.33:0/3651331486 learned_addr learned my addr 172.21.15.33:0/3651331486 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:05.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96cdb76700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96c8158d00 0x7f96c80c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 -- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8068990 msgr2=0x7f96c8147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8068990 0x7f96c8147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 -- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96c8158d00 msgr2=0x7f96c80c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.377 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96c8158d00 0x7f96c80c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 -- 172.21.15.33:0/3651331486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4009420 con 0x7f96c80cf420 2023-10-06T02:47:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ccb74700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 0x7f96c80b5180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c590 tx=0x7f96b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:05.378 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011590 con 0x7f96c80cf420 2023-10-06T02:47:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b8011710 con 0x7f96c80cf420 2023-10-06T02:47:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c80c76b0 con 0x7f96c80cf420 2023-10-06T02:47:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011ad0 con 0x7f96c80cf420 2023-10-06T02:47:05.379 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.368+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c80c7c70 con 0x7f96c80cf420 2023-10-06T02:47:05.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.372+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c80c8430 con 0x7f96c80cf420 2023-10-06T02:47:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.372+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f96b801b3b0 con 0x7f96c80cf420 2023-10-06T02:47:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.372+0000 7f96bd7fa700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96ac060550 0x7f96ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.372+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f96b807bc40 con 0x7f96c80cf420 2023-10-06T02:47:05.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.372+0000 7f96cd375700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96ac060550 0x7f96ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.372+0000 7f96cd375700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96ac060550 0x7f96ac062a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96c40093f0 tx=0x7f96c4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:05.383 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.376+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f96b804c290 con 0x7f96c80cf420 2023-10-06T02:47:05.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.540+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96c8071040 con 0x7f96c80cf420 2023-10-06T02:47:05.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96bd7fa700 1 -- 172.21.15.33:0/3651331486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13984 (secure 0 0 0) 0x7f96b8017020 con 0x7f96c80cf420 2023-10-06T02:47:05.548 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:05.548 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"fsid":"8ff12af2-25a8-465d-8914-d29bca58b4dd","created":"2023-10-06T02:44:58.351307+0000","modified":"2023-10-06T02:47:02.427992+0000","last_up_change":"2023-10-06T02:45:09.541017+0000","last_in_change":"2023-10-06T02:45:01.193156+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T02:45:10.622496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T02:45:16.980990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2b7ece44-9249-4d47-b37b-f1830b709f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":33421},{"type":"v1","addr":"172.21.15.33:6818","nonce":33421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":33421},{"type":"v1","addr":"172.21.15.33:6822","nonce":33421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":33421},{"type":"v1","addr":"172.21.15.33:6830","nonce":33421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":33421},{"type":"v1","addr":"172.21.15.33:6826","nonce":33421}]},"public_addr":"172.21.15.33:6818/33421","cluster_addr":"172.21.15.33:6822/33421","heartbeat_back_addr":"172.21.15.33:6830/33421","heartbeat_front_addr":"172.21.15.33:6826/33421","state":["exists","up"]},{"osd":1,"uuid":"6f8fa54f-9a5e-48ea-aaf8-f3d37db38da5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":33423},{"type":"v1","addr":"172.21.15.33:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":33423},{"type":"v1","addr":"172.21.15.33:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":33423},{"type":"v1","addr":"172.21.15.33:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":33423},{"type":"v1","addr":"172.21.15.33:6805","nonce":33423}]},"public_addr":"172.21.15.33:6801/33423","cluster_addr":"172.21.15.33:6803/33423","heartbeat_back_addr":"172.21.15.33:6807/33423","heartbeat_front_addr":"172.21.15.33:6805/33423","state":["exists","up"]},{"osd":2,"uuid":"43209fe6-de05-4f66-b8a2-cc5c6e484204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":33422},{"type":"v1","addr":"172.21.15.33:6809","nonce":33422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":33422},{"type":"v1","addr":"172.21.15.33:6811","nonce":33422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":33422},{"type":"v1","addr":"172.21.15.33:6815","nonce":33422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":33422},{"type":"v1","addr":"172.21.15.33:6813","nonce":33422}]},"public_addr":"172.21.15.33:6809/33422","cluster_addr":"172.21.15.33:6811/33422","heartbeat_back_addr":"172.21.15.33:6815/33422","heartbeat_front_addr":"172.21.15.33:6813/33422","state":["exists","up"]},{"osd":3,"uuid":"6ccf7bd9-d2ff-4456-ac25-6dbed4cde389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":33424},{"type":"v1","addr":"172.21.15.33:6819","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":33424},{"type":"v1","addr":"172.21.15.33:6823","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6829","nonce":33424},{"type":"v1","addr":"172.21.15.33:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6825","nonce":33424},{"type":"v1","addr":"172.21.15.33:6827","nonce":33424}]},"public_addr":"172.21.15.33:6819/33424","cluster_addr":"172.21.15.33:6823/33424","heartbeat_back_addr":"172.21.15.33:6831/33424","heartbeat_front_addr":"172.21.15.33:6827/33424","state":["exists","up"]},{"osd":4,"uuid":"e5b62b70-bdd9-43de-83ab-33415b01009c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":33657},{"type":"v1","addr":"172.21.15.167:6801","nonce":33657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":33657},{"type":"v1","addr":"172.21.15.167:6803","nonce":33657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":33657},{"type":"v1","addr":"172.21.15.167:6807","nonce":33657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":33657},{"type":"v1","addr":"172.21.15.167:6805","nonce":33657}]},"public_addr":"172.21.15.167:6801/33657","cluster_addr":"172.21.15.167:6803/33657","heartbeat_back_addr":"172.21.15.167:6807/33657","heartbeat_front_addr":"172.21.15.167:6805/33657","state":["exists","up"]},{"osd":5,"uuid":"0fbdfffa-3a66-49c0-8bc8-916e5f5bdf55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":33659},{"type":"v1","addr":"172.21.15.167:6809","nonce":33659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":33659},{"type":"v1","addr":"172.21.15.167:6811","nonce":33659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":33659},{"type":"v1","addr":"172.21.15.167:6815","nonce":33659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":33659},{"type":"v1","addr":"172.21.15.167:6813","nonce":33659}]},"public_addr":"172.21.15.167:6809/33659","cluster_addr":"172.21.15.167:6811/33659","heartbeat_back_addr":"172.21.15.167:6815/33659","heartbeat_front_addr":"172.21.15.167:6813/33659","state":["exists","up"]},{"osd":6,"uuid":"7ed3b4e3-0ea4-484c-a036-f9f734aa8f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":33656},{"type":"v1","addr":"172.21.15.167:6817","nonce":33656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":33656},{"type":"v1","addr":"172.21.15.167:6819","nonce":33656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":33656},{"type":"v1","addr":"172.21.15.167:6823","nonce":33656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":33656},{"type":"v1","addr":"172.21.15.167:6821","nonce":33656}]},"public_addr":"172.21.15.167:6817/33656","cluster_addr":"172.21.15.167:6819/33656","heartbeat_back_addr":"172.21.15.167:6823/33656","heartbeat_front_addr":"172.21.15.167:6821/33656","state":["exists","up"]},{"osd":7,"uuid":"f3695573-fd9c-49c3-b8ab-93ffdc80aced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":33658},{"type":"v1","addr":"172.21.15.167:6825","nonce":33658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":33658},{"type":"v1","addr":"172.21.15.167:6827","nonce":33658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":33658},{"type":"v1","addr":"172.21.15.167:6831","nonce":33658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":33658},{"type":"v1","addr":"172.21.15.167:6829","nonce":33658}]},"public_addr":"172.21.15.167:6825/33658","cluster_addr":"172.21.15.167:6827/33658","heartbeat_back_addr":"172.21.15.167:6831/33658","heartbeat_front_addr":"172.21.15.167:6829/33658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.387189+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.439669+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.372418+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.656741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.534458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.512679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.586523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T02:45:06.432231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6835/3594813995":"2023-10-07T02:46:59.054601+0000","172.21.15.33:6834/3594813995":"2023-10-07T02:46:59.054601+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96ac060550 msgr2=0x7f96ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96ac060550 0x7f96ac062a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96c40093f0 tx=0x7f96c4017000 comp rx=0 tx=0).stop 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 msgr2=0x7f96c80b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 0x7f96c80b5180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c590 tx=0x7f96b800c8a0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 shutdown_connections 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96c8158d00 0x7f96c80c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c80cf420 0x7f96c80b5180 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f96ac060550 0x7f96ac062a00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 --2- 172.21.15.33:0/3651331486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96c8068990 0x7f96c8147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 >> 172.21.15.33:0/3651331486 conn(0x7f96c8067a80 msgr2=0x7f96c815b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 shutdown_connections 2023-10-06T02:47:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:05.544+0000 7f96ce377700 1 -- 172.21.15.33:0/3651331486 wait complete. 2023-10-06T02:47:06.564 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T02:47:06.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:06.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.680+0000 7f515ccfa700 1 Processor -- start 2023-10-06T02:47:06.687 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.680+0000 7f515ccfa700 1 -- start start 2023-10-06T02:47:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5158158ab0 0x7f5158068490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51580689d0 0x7f515815c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515815af10 con 0x7f515815cd80 2023-10-06T02:47:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5158068e20 con 0x7f5158158ab0 2023-10-06T02:47:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5158159fe0 con 0x7f51580689d0 2023-10-06T02:47:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f5156ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51580689d0 0x7f515815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5158158ab0 0x7f5158068490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f5156ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51580689d0 0x7f515815c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:45262/0 (socket says 172.21.15.33:45262) 2023-10-06T02:47:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f5156ffd700 1 -- 172.21.15.33:0/943004629 learned_addr learned my addr 172.21.15.33:0/943004629 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:06.690 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5158158ab0 0x7f5158068490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:53660/0 (socket says 172.21.15.33:53660) 2023-10-06T02:47:06.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 -- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51580689d0 msgr2=0x7f515815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:06.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 --2- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51580689d0 0x7f515815c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 -- 172.21.15.33:0/943004629 >> v1:172.21.15.167:6789/0 conn(0x7f515815cd80 legacy=0x7f5158158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 -- 172.21.15.33:0/943004629 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515815a8c0 con 0x7f5158158ab0 2023-10-06T02:47:06.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51577fe700 1 --2- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5158158ab0 0x7f5158068490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5148009e60 tx=0x7f5148009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649cafa5c32e25c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.693 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51567fc700 1 -- 172.21.15.33:0/943004629 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800e070 con 0x7f5158158ab0 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51567fc700 1 -- 172.21.15.33:0/943004629 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514800bb60 con 0x7f5158158ab0 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f51567fc700 1 -- 172.21.15.33:0/943004629 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148012430 con 0x7f5158158ab0 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 -- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5158158ab0 msgr2=0x7f5158068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 --2- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5158158ab0 0x7f5158068490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5148009e60 tx=0x7f5148009780 comp rx=0 tx=0).stop 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 -- 172.21.15.33:0/943004629 shutdown_connections 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 --2- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51580689d0 0x7f515815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 --2- 172.21.15.33:0/943004629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5158158ab0 0x7f5158068490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.694 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.684+0000 7f515ccfa700 1 -- 172.21.15.33:0/943004629 >> 172.21.15.33:0/943004629 conn(0x7f5158067a80 msgr2=0x7f51580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:06.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 -- 172.21.15.33:0/943004629 shutdown_connections 2023-10-06T02:47:06.695 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 -- 172.21.15.33:0/943004629 wait complete. 2023-10-06T02:47:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 Processor -- start 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 -- start start 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580689d0 0x7f5158169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5158158ab0 0x7f51580af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515815cd80 0x7f51581522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51580b2cc0 con 0x7f51580689d0 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51580b2990 con 0x7f515815cd80 2023-10-06T02:47:06.699 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f515ccfa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580afe80 con 0x7f5158158ab0 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580689d0 0x7f5158169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580689d0 0x7f5158169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50594/0 (socket says 172.21.15.33:50594) 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f5156ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5158158ab0 0x7f51580af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 -- 172.21.15.33:0/3803939693 learned_addr learned my addr 172.21.15.33:0/3803939693 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f5157fff700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515815cd80 0x7f51581522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 -- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5158158ab0 msgr2=0x7f51580af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5158158ab0 0x7f51580af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 -- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515815cd80 msgr2=0x7f51581522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f515815cd80 0x7f51581522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.688+0000 7f51577fe700 1 -- 172.21.15.33:0/3803939693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5148009a20 con 0x7f51580689d0 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f51577fe700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580689d0 0x7f5158169780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f51480097c0 tx=0x7f5148019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.700 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148012c40 con 0x7f51580689d0 2023-10-06T02:47:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5148012da0 con 0x7f51580689d0 2023-10-06T02:47:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51580b0150 con 0x7f51580689d0 2023-10-06T02:47:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480276c0 con 0x7f51580689d0 2023-10-06T02:47:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51580b0710 con 0x7f51580689d0 2023-10-06T02:47:06.701 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5158071040 con 0x7f51580689d0 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.692+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5148024070 con 0x7f51580689d0 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f514f7fe700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5138060550 0x7f5138062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f514807ecd0 con 0x7f51580689d0 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f514f7fe700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7f5138066e60 0x7f5138069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5138069960 con 0x7f5138066e60 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5148081310 con 0x7f51580689d0 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f5156ffd700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5138060550 0x7f5138062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.703 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f5157fff700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7f5138066e60 0x7f5138069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f5156ffd700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5138060550 0x7f5138062a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5158158220 tx=0x7f5140009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f5157fff700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7f5138066e60 0x7f5138069290 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.704 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.696+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== osd.0 v2:172.21.15.33:6816/33421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f5138069960 con 0x7f5138066e60 2023-10-06T02:47:06.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.712+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 --> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f51580693c0 con 0x7f5138066e60 2023-10-06T02:47:06.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f514f7fe700 1 -- 172.21.15.33:0/3803939693 <== osd.0 v2:172.21.15.33:6816/33421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f51580693c0 con 0x7f5138066e60 2023-10-06T02:47:06.720 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:06.720 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:06.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7f5138066e60 msgr2=0x7f5138069290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:6816/33421,v1:172.21.15.33:6818/33421] conn(0x7f5138066e60 0x7f5138069290 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.721 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5138060550 msgr2=0x7f5138062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5138060550 0x7f5138062a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5158158220 tx=0x7f5140009250 comp rx=0 tx=0).stop 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580689d0 msgr2=0x7f5158169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 --2- 172.21.15.33:0/3803939693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51580689d0 0x7f5158169780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f51480097c0 tx=0x7f5148019000 comp rx=0 tx=0).stop 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f5157fff700 1 -- 172.21.15.33:0/3803939693 reap_dead start 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 shutdown_connections 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 >> 172.21.15.33:0/3803939693 conn(0x7f5158067a80 msgr2=0x7f51581549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 shutdown_connections 2023-10-06T02:47:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.716+0000 7f515ccfa700 1 -- 172.21.15.33:0/3803939693 wait complete. 2023-10-06T02:47:06.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T02:47:06.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.848+0000 7f274c683700 1 Processor -- start 2023-10-06T02:47:06.855 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.848+0000 7f274c683700 1 -- start start 2023-10-06T02:47:06.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2744142590 0x7f2744142960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.856 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27440b2b70 0x7f27440cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440cfc10 con 0x7f27440684b0 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27440b2ff0 con 0x7f27440b2b70 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27440b3250 con 0x7f2744142590 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27440b2b70 0x7f27440cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27440b2b70 0x7f27440cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50596/0 (socket says 172.21.15.33:50596) 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274a41f700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2744142590 0x7f2744142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.857 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 -- 172.21.15.33:0/1670525756 learned_addr learned my addr 172.21.15.33:0/1670525756 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 -- 172.21.15.33:0/1670525756 >> v2:172.21.15.167:3300/0 conn(0x7f2744142590 msgr2=0x7f2744142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:06.858 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 --2- 172.21.15.33:0/1670525756 >> v2:172.21.15.167:3300/0 conn(0x7f2744142590 0x7f2744142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 -- 172.21.15.33:0/1670525756 >> v1:172.21.15.167:6789/0 conn(0x7f27440684b0 legacy=0x7f274415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 -- 172.21.15.33:0/1670525756 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440b3530 con 0x7f27440b2b70 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274a41f700 1 --2- 172.21.15.33:0/1670525756 >> v2:172.21.15.167:3300/0 conn(0x7f2744142590 0x7f2744142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f2749c1e700 1 --2- 172.21.15.33:0/1670525756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440b2b70 0x7f27440cf6d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f27380099b0 tx=0x7f2738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8105458d01cbef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274941d700 1 -- 172.21.15.33:0/1670525756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738009ea0 con 0x7f27440b2b70 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274941d700 1 -- 172.21.15.33:0/1670525756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273800b820 con 0x7f27440b2b70 2023-10-06T02:47:06.859 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274941d700 1 -- 172.21.15.33:0/1670525756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800bbc0 con 0x7f27440b2b70 2023-10-06T02:47:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- 172.21.15.33:0/1670525756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440b2b70 msgr2=0x7f27440cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 --2- 172.21.15.33:0/1670525756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440b2b70 0x7f27440cf6d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f27380099b0 tx=0x7f2738009250 comp rx=0 tx=0).stop 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- 172.21.15.33:0/1670525756 shutdown_connections 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 --2- 172.21.15.33:0/1670525756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440b2b70 0x7f27440cf6d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 --2- 172.21.15.33:0/1670525756 >> v2:172.21.15.167:3300/0 conn(0x7f2744142590 0x7f2744142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- 172.21.15.33:0/1670525756 >> 172.21.15.33:0/1670525756 conn(0x7f27440d0830 msgr2=0x7f27440b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- 172.21.15.33:0/1670525756 shutdown_connections 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.852+0000 7f274c683700 1 -- 172.21.15.33:0/1670525756 wait complete. 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 Processor -- start 2023-10-06T02:47:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 -- start start 2023-10-06T02:47:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 0x7f2744158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27440b2b70 0x7f2744169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744142590 0x7f27440afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274413bb30 con 0x7f27440684b0 2023-10-06T02:47:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2744158e60 con 0x7f2744142590 2023-10-06T02:47:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2744169fb0 con 0x7f27440b2b70 2023-10-06T02:47:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 0x7f2744158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 0x7f2744158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50608/0 (socket says 172.21.15.33:50608) 2023-10-06T02:47:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 -- 172.21.15.33:0/3262042589 learned_addr learned my addr 172.21.15.33:0/3262042589 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f2749c1e700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27440b2b70 0x7f2744169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274ac20700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744142590 0x7f27440afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 -- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27440b2b70 msgr2=0x7f2744169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27440b2b70 0x7f2744169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 -- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744142590 msgr2=0x7f27440afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744142590 0x7f27440afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 -- 172.21.15.33:0/3262042589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c009420 con 0x7f27440684b0 2023-10-06T02:47:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274a41f700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 0x7f2744158920 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f273c00ec80 tx=0x7f273c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00c850 con 0x7f27440684b0 2023-10-06T02:47:06.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c00c9b0 con 0x7f27440684b0 2023-10-06T02:47:06.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27380094f0 con 0x7f27440684b0 2023-10-06T02:47:06.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00cd30 con 0x7f27440684b0 2023-10-06T02:47:06.868 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.856+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274416a570 con 0x7f27440684b0 2023-10-06T02:47:06.869 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.860+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f273c013680 con 0x7f27440684b0 2023-10-06T02:47:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.860+0000 7f27427fc700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2730060550 0x7f2730062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.860+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27440710b0 con 0x7f27440684b0 2023-10-06T02:47:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.860+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f273c07ca40 con 0x7f27440684b0 2023-10-06T02:47:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.860+0000 7f2749c1e700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2730060550 0x7f2730062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.860+0000 7f2749c1e700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2730060550 0x7f2730062a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27380099b0 tx=0x7f2738017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:06.873 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:06.868+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f273c050b20 con 0x7f27440684b0 2023-10-06T02:47:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.032+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f274413bc70 con 0x7f2730060550 2023-10-06T02:47:07.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f27427fc700 1 -- 172.21.15.33:0/3262042589 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f273800b990 con 0x7f2730060550 2023-10-06T02:47:07.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2730060550 msgr2=0x7f2730062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2730060550 0x7f2730062a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27380099b0 tx=0x7f2738017000 comp rx=0 tx=0).stop 2023-10-06T02:47:07.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 msgr2=0x7f2744158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 0x7f2744158920 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f273c00ec80 tx=0x7f273c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 shutdown_connections 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2730060550 0x7f2730062a00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27440b2b70 0x7f2744169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2744142590 0x7f27440afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 --2- 172.21.15.33:0/3262042589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27440684b0 0x7f2744158920 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 >> 172.21.15.33:0/3262042589 conn(0x7f27440d0830 msgr2=0x7f274415bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 shutdown_connections 2023-10-06T02:47:07.044 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.036+0000 7f274c683700 1 -- 172.21.15.33:0/3262042589 wait complete. 2023-10-06T02:47:07.044 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T02:47:07.055 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T02:47:07.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:07.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.160+0000 7fb573692700 1 Processor -- start 2023-10-06T02:47:07.167 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.160+0000 7fb573692700 1 -- start start 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb56c0c8d70 0x7fb56c067da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56c15cb60 con 0x7fb56c0682e0 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c0c77e0 con 0x7fb56c0c8d70 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0c7ac0 con 0x7fb56c158d50 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb571c2f700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb56c0682e0 0x7fb56c06c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45082/0 (socket says 172.21.15.33:45082) 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb571c2f700 1 -- 172.21.15.33:0/3652083808 learned_addr learned my addr 172.21.15.33:0/3652083808 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb570c2d700 1 --2- 172.21.15.33:0/3652083808 >> v2:172.21.15.167:3300/0 conn(0x7fb56c0c8d70 0x7fb56c067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb57142e700 1 --2- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb563fff700 1 -- 172.21.15.33:0/3652083808 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267938921 0 0) 0x7fb56c15cb60 con 0x7fb56c0682e0 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb563fff700 1 -- 172.21.15.33:0/3652083808 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb558003570 con 0x7fb56c0682e0 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb570c2d700 1 -- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 msgr2=0x7fb56c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb570c2d700 1 --2- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb570c2d700 1 -- 172.21.15.33:0/3652083808 >> v1:172.21.15.33:6789/0 conn(0x7fb56c0682e0 legacy=0x7fb56c06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb570c2d700 1 -- 172.21.15.33:0/3652083808 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0c7da0 con 0x7fb56c0c8d70 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb570c2d700 1 --2- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0c8d70 0x7fb56c067da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0098e0 tx=0x7fb55c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ce7452bd3785d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb563fff700 1 -- 172.21.15.33:0/3652083808 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011630 con 0x7fb56c0c8d70 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb563fff700 1 -- 172.21.15.33:0/3652083808 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c011790 con 0x7fb56c0c8d70 2023-10-06T02:47:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb563fff700 1 -- 172.21.15.33:0/3652083808 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011b30 con 0x7fb56c0c8d70 2023-10-06T02:47:07.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0c8d70 msgr2=0x7fb56c067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.172 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 --2- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0c8d70 0x7fb56c067da0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0098e0 tx=0x7fb55c0091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- 172.21.15.33:0/3652083808 shutdown_connections 2023-10-06T02:47:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 --2- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0c8d70 0x7fb56c067da0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 --2- 172.21.15.33:0/3652083808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- 172.21.15.33:0/3652083808 >> 172.21.15.33:0/3652083808 conn(0x7fb56c0c96a0 msgr2=0x7fb56c0c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- 172.21.15.33:0/3652083808 shutdown_connections 2023-10-06T02:47:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.164+0000 7fb573692700 1 -- 172.21.15.33:0/3652083808 wait complete. 2023-10-06T02:47:07.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 Processor -- start 2023-10-06T02:47:07.174 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 -- start start 2023-10-06T02:47:07.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0682e0 0x7fb56c16a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0b1820 0x7fb56c1692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c159530 con 0x7fb56c0682e0 2023-10-06T02:47:07.177 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c16aa00 con 0x7fb56c0b1820 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb573692700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c169c40 con 0x7fb56c158d50 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0682e0 0x7fb56c16a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0682e0 0x7fb56c16a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50610/0 (socket says 172.21.15.33:50610) 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 -- 172.21.15.33:0/1261804108 learned_addr learned my addr 172.21.15.33:0/1261804108 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb570c2d700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb571c2f700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0b1820 0x7fb56c1692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 -- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 msgr2=0x7fb56c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 -- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0b1820 msgr2=0x7fb56c1692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0b1820 0x7fb56c1692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb57142e700 1 -- 172.21.15.33:0/1261804108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c009420 con 0x7fb56c0682e0 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb571c2f700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb56c0b1820 0x7fb56c1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.168+0000 7fb570c2d700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb56c158d50 0x7fb56c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb57142e700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0682e0 0x7fb56c16a4c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e930 tx=0x7fb5680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800eed0 con 0x7fb56c0682e0 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56800c850 con 0x7fb56c0682e0 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c169f20 con 0x7fb56c0682e0 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c1591c0 con 0x7fb56c0682e0 2023-10-06T02:47:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800cbf0 con 0x7fb56c0682e0 2023-10-06T02:47:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb54c000fd0 con 0x7fb56c0682e0 2023-10-06T02:47:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb568013470 con 0x7fb56c0682e0 2023-10-06T02:47:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb550060550 0x7fb550062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fb56807c900 con 0x7fb56c0682e0 2023-10-06T02:47:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.172+0000 7fb5617fa700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7fb550066d70 0x7fb5500691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 --> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb550069870 con 0x7fb550066d70 2023-10-06T02:47:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb568081240 con 0x7fb56c0682e0 2023-10-06T02:47:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb570c2d700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb550060550 0x7fb550062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb571c2f700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7fb550066d70 0x7fb5500691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb571c2f700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7fb550066d70 0x7fb5500691a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb570c2d700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb550060550 0x7fb550062a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0c9140 tx=0x7fb55c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.176+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== osd.1 v2:172.21.15.33:6800/33423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb550069870 con 0x7fb550066d70 2023-10-06T02:47:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.192+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 --> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb54c002d50 con 0x7fb550066d70 2023-10-06T02:47:07.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb5617fa700 1 -- 172.21.15.33:0/1261804108 <== osd.1 v2:172.21.15.33:6800/33423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb54c002d50 con 0x7fb550066d70 2023-10-06T02:47:07.201 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:07.201 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:07.202 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7fb550066d70 msgr2=0x7fb5500691a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:6800/33423,v1:172.21.15.33:6801/33423] conn(0x7fb550066d70 0x7fb5500691a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb550060550 msgr2=0x7fb550062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fb550060550 0x7fb550062a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0c9140 tx=0x7fb55c017000 comp rx=0 tx=0).stop 2023-10-06T02:47:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0682e0 msgr2=0x7fb56c16a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.203 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 --2- 172.21.15.33:0/1261804108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0682e0 0x7fb56c16a4c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb56800e930 tx=0x7fb5680091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb571c2f700 1 -- 172.21.15.33:0/1261804108 reap_dead start 2023-10-06T02:47:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 shutdown_connections 2023-10-06T02:47:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 >> 172.21.15.33:0/1261804108 conn(0x7fb56c0c96a0 msgr2=0x7fb56c0ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 shutdown_connections 2023-10-06T02:47:07.204 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.196+0000 7fb573692700 1 -- 172.21.15.33:0/1261804108 wait complete. 2023-10-06T02:47:07.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T02:47:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.316+0000 7f6fa631e700 1 Processor -- start 2023-10-06T02:47:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.316+0000 7f6fa631e700 1 -- start start 2023-10-06T02:47:07.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6fa0154e90 0x7f6fa00c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.324 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6fa0155260 0x7f6fa00c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa00c7d30 con 0x7f6fa0067840 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa0155690 con 0x7f6fa0155260 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa00c81c0 con 0x7f6fa0154e90 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa48bb700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6fa0067840 0x7f6fa0158e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45092/0 (socket says 172.21.15.33:45092) 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f97fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6fa0155260 0x7f6fa00c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa48bb700 1 -- 172.21.15.33:0/2252719744 learned_addr learned my addr 172.21.15.33:0/2252719744 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f9ffff700 1 --2- 172.21.15.33:0/2252719744 >> v2:172.21.15.167:3300/0 conn(0x7f6fa0154e90 0x7f6fa00c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f9f7fe700 1 -- 172.21.15.33:0/2252719744 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803603070 0 0) 0x7f6fa00c7d30 con 0x7f6fa0067840 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f97fff700 1 -- 172.21.15.33:0/2252719744 >> v2:172.21.15.167:3300/0 conn(0x7f6fa0154e90 msgr2=0x7f6fa00c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f97fff700 1 --2- 172.21.15.33:0/2252719744 >> v2:172.21.15.167:3300/0 conn(0x7f6fa0154e90 0x7f6fa00c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f97fff700 1 -- 172.21.15.33:0/2252719744 >> v1:172.21.15.33:6789/0 conn(0x7f6fa0067840 legacy=0x7f6fa0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f97fff700 1 -- 172.21.15.33:0/2252719744 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa00c7120 con 0x7f6fa0155260 2023-10-06T02:47:07.326 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f97fff700 1 --2- 172.21.15.33:0/2252719744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00c1380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0064400 tx=0x7f6f880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecafded1e9dfa39f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f9f7fe700 1 -- 172.21.15.33:0/2252719744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800b6b0 con 0x7f6fa0155260 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f9f7fe700 1 -- 172.21.15.33:0/2252719744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800b810 con 0x7f6fa0155260 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6f9f7fe700 1 -- 172.21.15.33:0/2252719744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800bbb0 con 0x7f6fa0155260 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- 172.21.15.33:0/2252719744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 msgr2=0x7f6fa00c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 --2- 172.21.15.33:0/2252719744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00c1380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0064400 tx=0x7f6f880091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- 172.21.15.33:0/2252719744 shutdown_connections 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 --2- 172.21.15.33:0/2252719744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00c1380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.327 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 --2- 172.21.15.33:0/2252719744 >> v2:172.21.15.167:3300/0 conn(0x7f6fa0154e90 0x7f6fa00c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- 172.21.15.33:0/2252719744 >> 172.21.15.33:0/2252719744 conn(0x7f6fa00c83f0 msgr2=0x7f6fa00c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- 172.21.15.33:0/2252719744 shutdown_connections 2023-10-06T02:47:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.320+0000 7f6fa631e700 1 -- 172.21.15.33:0/2252719744 wait complete. 2023-10-06T02:47:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 Processor -- start 2023-10-06T02:47:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 -- start start 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fa0067840 0x7f6fa00c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0154e90 0x7f6fa01686c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00adac0 con 0x7f6fa0155260 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0168c00 con 0x7f6fa0154e90 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00ad560 con 0x7f6fa0067840 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50636/0 (socket says 172.21.15.33:50636) 2023-10-06T02:47:07.330 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 -- 172.21.15.33:0/3058402299 learned_addr learned my addr 172.21.15.33:0/3058402299 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6f97fff700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0154e90 0x7f6fa01686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6f9ffff700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fa0067840 0x7f6fa00c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 -- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fa0067840 msgr2=0x7f6fa00c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fa0067840 0x7f6fa00c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 -- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0154e90 msgr2=0x7f6fa01686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0154e90 0x7f6fa01686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 -- 172.21.15.33:0/3058402299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009420 con 0x7f6fa0155260 2023-10-06T02:47:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa48bb700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00ad020 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800cc20 tx=0x7f6f980019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800a870 con 0x7f6fa0155260 2023-10-06T02:47:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9800a9d0 con 0x7f6fa0155260 2023-10-06T02:47:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa00ad7e0 con 0x7f6fa0155260 2023-10-06T02:47:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800ad90 con 0x7f6fa0155260 2023-10-06T02:47:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.324+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa00a9100 con 0x7f6fa0155260 2023-10-06T02:47:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.328+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa00a8c80 con 0x7f6fa0155260 2023-10-06T02:47:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.328+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6f98007300 con 0x7f6fa0155260 2023-10-06T02:47:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.328+0000 7f6f9cff9700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6f80060550 0x7f6f80062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.328+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f6f9807c9e0 con 0x7f6fa0155260 2023-10-06T02:47:07.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.328+0000 7f6f9ffff700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6f80060550 0x7f6f80062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.336 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.332+0000 7f6f9ffff700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6f80060550 0x7f6f80062a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.340+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6fa00a8c80 con 0x7f6fa0155260 2023-10-06T02:47:07.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.500+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fa00a8c80 con 0x7f6f80060550 2023-10-06T02:47:07.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.500+0000 7f6f9cff9700 1 -- 172.21.15.33:0/3058402299 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6fa00a8c80 con 0x7f6f80060550 2023-10-06T02:47:07.509 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6f80060550 msgr2=0x7f6f80062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6f80060550 0x7f6f80062a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90008040 comp rx=0 tx=0).stop 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 msgr2=0x7f6fa00ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00ad020 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800cc20 tx=0x7f6f980019b0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 shutdown_connections 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa0155260 0x7f6fa00ad020 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fa0154e90 0x7f6fa01686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f6f80060550 0x7f6f80062a00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 --2- 172.21.15.33:0/3058402299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fa0067840 0x7f6fa00c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 >> 172.21.15.33:0/3058402299 conn(0x7f6fa00c83f0 msgr2=0x7f6fa00c1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 shutdown_connections 2023-10-06T02:47:07.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.504+0000 7f6fa631e700 1 -- 172.21.15.33:0/3058402299 wait complete. 2023-10-06T02:47:07.511 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T02:47:07.522 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T02:47:07.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:07.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.632+0000 7f218c942700 1 Processor -- start 2023-10-06T02:47:07.639 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.632+0000 7f218c942700 1 -- start start 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218c942700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2188154ee0 0x7f21881552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218c942700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21880aeb10 con 0x7f218806e430 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218c942700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21880c6600 con 0x7f21880c76a0 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218c942700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21880c68e0 con 0x7f2188154ee0 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218759e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f218806e430 0x7f21880ac4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45104/0 (socket says 172.21.15.33:45104) 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218759e700 1 -- 172.21.15.33:0/3330957993 learned_addr learned my addr 172.21.15.33:0/3330957993 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f2186d9d700 1 --2- 172.21.15.33:0/3330957993 >> v2:172.21.15.167:3300/0 conn(0x7f2188154ee0 0x7f21881552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380187745 0 0) 0x7f21880aeb10 con 0x7f218806e430 2023-10-06T02:47:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2174003570 con 0x7f218806e430 2023-10-06T02:47:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957308264 0 0) 0x7f21880c6600 con 0x7f21880c76a0 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21880aeb10 con 0x7f21880c76a0 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825128432 0 0) 0x7f2174003570 con 0x7f218806e430 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f2186d9d700 1 -- 172.21.15.33:0/3330957993 >> v1:172.21.15.167:6789/0 conn(0x7f21880c76a0 legacy=0x7f218806dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f2186d9d700 1 -- 172.21.15.33:0/3330957993 >> v1:172.21.15.33:6789/0 conn(0x7f218806e430 legacy=0x7f21880ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f2186d9d700 1 -- 172.21.15.33:0/3330957993 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21880c6bc0 con 0x7f2188154ee0 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f2186d9d700 1 --2- 172.21.15.33:0/3330957993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 0x7f21881552b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21780098e0 tx=0x7f21780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d602873527eea054 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21800017f0 con 0x7f2188154ee0 2023-10-06T02:47:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178011420 con 0x7f2188154ee0 2023-10-06T02:47:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.636+0000 7f218659c700 1 -- 172.21.15.33:0/3330957993 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21780117c0 con 0x7f2188154ee0 2023-10-06T02:47:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 -- 172.21.15.33:0/3330957993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 msgr2=0x7f21881552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 --2- 172.21.15.33:0/3330957993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 0x7f21881552b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21780098e0 tx=0x7f21780091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 -- 172.21.15.33:0/3330957993 shutdown_connections 2023-10-06T02:47:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 --2- 172.21.15.33:0/3330957993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 0x7f21881552b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 -- 172.21.15.33:0/3330957993 >> 172.21.15.33:0/3330957993 conn(0x7f21880ccb30 msgr2=0x7f21880c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 -- 172.21.15.33:0/3330957993 shutdown_connections 2023-10-06T02:47:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 -- 172.21.15.33:0/3330957993 wait complete. 2023-10-06T02:47:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 Processor -- start 2023-10-06T02:47:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.640+0000 7f218c942700 1 -- start start 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f218806e430 0x7f21880a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21880c76a0 0x7f2188157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 0x7f218816a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21880a9b60 con 0x7f21880c76a0 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21881581f0 con 0x7f2188154ee0 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21880a8f50 con 0x7f218806e430 2023-10-06T02:47:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21880c76a0 0x7f2188157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21880c76a0 0x7f2188157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50638/0 (socket says 172.21.15.33:50638) 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 -- 172.21.15.33:0/3164074186 learned_addr learned my addr 172.21.15.33:0/3164074186 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218759e700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 0x7f218816a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f2186d9d700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f218806e430 0x7f21880a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 -- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f218806e430 msgr2=0x7f21880a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f218806e430 0x7f21880a88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 -- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 msgr2=0x7f218816a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2188154ee0 0x7f218816a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 -- 172.21.15.33:0/3164074186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178009420 con 0x7f21880c76a0 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f2186d9d700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f218806e430 0x7f21880a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217ffff700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21880c76a0 0x7f2188157cb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f217000c790 tx=0x7f2170007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170010510 con 0x7f21880c76a0 2023-10-06T02:47:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170010690 con 0x7f21880c76a0 2023-10-06T02:47:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21880a9230 con 0x7f21880c76a0 2023-10-06T02:47:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170010a50 con 0x7f21880c76a0 2023-10-06T02:47:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21880a97f0 con 0x7f21880c76a0 2023-10-06T02:47:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.644+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2188071040 con 0x7f21880c76a0 2023-10-06T02:47:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2170025430 con 0x7f21880c76a0 2023-10-06T02:47:07.654 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2168060550 0x7f2168062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f217007cff0 con 0x7f21880c76a0 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f2168066e60 0x7f2168069290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2168069960 con 0x7f2168066e60 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f217007d2e0 con 0x7f21880c76a0 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f218759e700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f2168066e60 0x7f2168069290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f2186d9d700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2168060550 0x7f2168062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f218759e700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f2168066e60 0x7f2168069290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.655 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f2186d9d700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2168060550 0x7f2168062a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f21880cb690 tx=0x7f2178016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.656 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.648+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== osd.2 v2:172.21.15.33:6808/33422 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2168069960 con 0x7f2168066e60 2023-10-06T02:47:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 --> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f21880c6ea0 con 0x7f2168066e60 2023-10-06T02:47:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f217effd700 1 -- 172.21.15.33:0/3164074186 <== osd.2 v2:172.21.15.33:6808/33422 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f21880c6ea0 con 0x7f2168066e60 2023-10-06T02:47:07.674 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:07.674 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:07.674 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f2168066e60 msgr2=0x7f2168069290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:6808/33422,v1:172.21.15.33:6809/33422] conn(0x7f2168066e60 0x7f2168069290 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2168060550 msgr2=0x7f2168062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2168060550 0x7f2168062a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f21880cb690 tx=0x7f2178016000 comp rx=0 tx=0).stop 2023-10-06T02:47:07.675 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21880c76a0 msgr2=0x7f2188157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 --2- 172.21.15.33:0/3164074186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21880c76a0 0x7f2188157cb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f217000c790 tx=0x7f2170007e10 comp rx=0 tx=0).stop 2023-10-06T02:47:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218759e700 1 -- 172.21.15.33:0/3164074186 reap_dead start 2023-10-06T02:47:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 shutdown_connections 2023-10-06T02:47:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 >> 172.21.15.33:0/3164074186 conn(0x7f21880ccb30 msgr2=0x7f21881590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 shutdown_connections 2023-10-06T02:47:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.668+0000 7f218c942700 1 -- 172.21.15.33:0/3164074186 wait complete. 2023-10-06T02:47:07.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T02:47:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 Processor -- start 2023-10-06T02:47:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- start start 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9e8150dc0 0x7fd9e8151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd9e80b20e0 0x7fd9e80cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e80caae0 con 0x7fd9e806bde0 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e806cd80 con 0x7fd9e8150dc0 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e806c7c0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ee362700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9e8150dc0 0x7fd9e8151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9edb61700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd9e806bde0 0x7fd9e80b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45114/0 (socket says 172.21.15.33:45114) 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9edb61700 1 -- 172.21.15.33:0/494633421 learned_addr learned my addr 172.21.15.33:0/494633421 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9eeb63700 1 --2- 172.21.15.33:0/494633421 >> v2:172.21.15.167:3300/0 conn(0x7fd9e80b20e0 0x7fd9e80cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ed360700 1 -- 172.21.15.33:0/494633421 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291675656 0 0) 0x7fd9e80caae0 con 0x7fd9e806bde0 2023-10-06T02:47:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ed360700 1 -- 172.21.15.33:0/494633421 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d0003570 con 0x7fd9e806bde0 2023-10-06T02:47:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ee362700 1 -- 172.21.15.33:0/494633421 >> v2:172.21.15.167:3300/0 conn(0x7fd9e80b20e0 msgr2=0x7fd9e80cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ee362700 1 --2- 172.21.15.33:0/494633421 >> v2:172.21.15.167:3300/0 conn(0x7fd9e80b20e0 0x7fd9e80cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ee362700 1 -- 172.21.15.33:0/494633421 >> v1:172.21.15.33:6789/0 conn(0x7fd9e806bde0 legacy=0x7fd9e80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ee362700 1 -- 172.21.15.33:0/494633421 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e806caa0 con 0x7fd9e8150dc0 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ee362700 1 --2- 172.21.15.33:0/494633421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e8150dc0 0x7fd9e8151190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b29d0 tx=0x7fd9d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f605b21609161f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ed360700 1 -- 172.21.15.33:0/494633421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011630 con 0x7fd9e8150dc0 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ed360700 1 -- 172.21.15.33:0/494633421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8011790 con 0x7fd9e8150dc0 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ed360700 1 -- 172.21.15.33:0/494633421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011b30 con 0x7fd9e8150dc0 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- 172.21.15.33:0/494633421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e8150dc0 msgr2=0x7fd9e8151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 --2- 172.21.15.33:0/494633421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e8150dc0 0x7fd9e8151190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b29d0 tx=0x7fd9d80091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- 172.21.15.33:0/494633421 shutdown_connections 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 --2- 172.21.15.33:0/494633421 >> v2:172.21.15.167:3300/0 conn(0x7fd9e80b20e0 0x7fd9e80cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 --2- 172.21.15.33:0/494633421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e8150dc0 0x7fd9e8151190 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- 172.21.15.33:0/494633421 >> 172.21.15.33:0/494633421 conn(0x7fd9e806e6c0 msgr2=0x7fd9e806ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- 172.21.15.33:0/494633421 shutdown_connections 2023-10-06T02:47:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.792+0000 7fd9ef364700 1 -- 172.21.15.33:0/494633421 wait complete. 2023-10-06T02:47:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 Processor -- start 2023-10-06T02:47:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 -- start start 2023-10-06T02:47:07.804 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e806bde0 0x7fd9e81545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 0x7fd9e80a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e8150dc0 0x7fd9e80a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8167da0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80a4f20 con 0x7fd9e806bde0 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9ef364700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80a45f0 con 0x7fd9e8150dc0 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9edb61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 0x7fd9e80a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9edb61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 0x7fd9e80a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50652/0 (socket says 172.21.15.33:50652) 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.796+0000 7fd9edb61700 1 -- 172.21.15.33:0/1953146220 learned_addr learned my addr 172.21.15.33:0/1953146220 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9ee362700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e806bde0 0x7fd9e81545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9eeb63700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e8150dc0 0x7fd9e80a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9edb61700 1 -- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e8150dc0 msgr2=0x7fd9e80a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9edb61700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e8150dc0 0x7fd9e80a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9edb61700 1 -- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e806bde0 msgr2=0x7fd9e81545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9edb61700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e806bde0 0x7fd9e81545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9edb61700 1 -- 172.21.15.33:0/1953146220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80093c0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9ee362700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e806bde0 0x7fd9e81545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9edb61700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 0x7fd9e80a5d60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400cc40 tx=0x7fd9e4001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e400a9c0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e400ab40 con 0x7fd9e80b20e0 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e80a48d0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40070f0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e80a53b0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e80a50b0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.807 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.800+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd9e4007520 con 0x7fd9e80b20e0 2023-10-06T02:47:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.804+0000 7fd9de7fc700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9d4060580 0x7fd9d4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.804+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fd9e407d6b0 con 0x7fd9e80b20e0 2023-10-06T02:47:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.804+0000 7fd9ee362700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9d4060580 0x7fd9d4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.804+0000 7fd9ee362700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9d4060580 0x7fd9d4062a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009390 tx=0x7fd9d8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:07.814 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.808+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd9e404dd60 con 0x7fd9e80b20e0 2023-10-06T02:47:07.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.972+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9e8071040 con 0x7fd9d4060580 2023-10-06T02:47:07.979 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.972+0000 7fd9de7fc700 1 -- 172.21.15.33:0/1953146220 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd9e8071040 con 0x7fd9d4060580 2023-10-06T02:47:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9d4060580 msgr2=0x7fd9d4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9d4060580 0x7fd9d4062a30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009390 tx=0x7fd9d8017000 comp rx=0 tx=0).stop 2023-10-06T02:47:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 msgr2=0x7fd9e80a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 0x7fd9e80a5d60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400cc40 tx=0x7fd9e4001a70 comp rx=0 tx=0).stop 2023-10-06T02:47:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 shutdown_connections 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fd9d4060580 0x7fd9d4062a30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9e8150dc0 0x7fd9e80a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9e80b20e0 0x7fd9e80a5d60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 --2- 172.21.15.33:0/1953146220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9e806bde0 0x7fd9e81545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 >> 172.21.15.33:0/1953146220 conn(0x7fd9e806e6c0 msgr2=0x7fd9e80aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 shutdown_connections 2023-10-06T02:47:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:07.980+0000 7fd9ef364700 1 -- 172.21.15.33:0/1953146220 wait complete. 2023-10-06T02:47:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T02:47:07.999 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T02:47:07.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 Processor -- start 2023-10-06T02:47:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 -- start start 2023-10-06T02:47:08.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2d100683c0 0x7f2d100c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.114 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2d10068790 0x7f2d1006a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100c5ea0 con 0x7f2d100c7480 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100c6180 con 0x7f2d100683c0 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0f59e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100c6460 con 0x7f2d10068790 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2d100683c0 0x7f2d100c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2d100683c0 0x7f2d100c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50668/0 (socket says 172.21.15.33:50668) 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 -- 172.21.15.33:0/1693805292 learned_addr learned my addr 172.21.15.33:0/1693805292 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0dd9b700 1 --2- 172.21.15.33:0/1693805292 >> v2:172.21.15.167:3300/0 conn(0x7f2d10068790 0x7f2d1006a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 -- 172.21.15.33:0/1693805292 >> v2:172.21.15.167:3300/0 conn(0x7f2d10068790 msgr2=0x7f2d1006a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 --2- 172.21.15.33:0/1693805292 >> v2:172.21.15.167:3300/0 conn(0x7f2d10068790 0x7f2d1006a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 -- 172.21.15.33:0/1693805292 >> v1:172.21.15.167:6789/0 conn(0x7f2d100c7480 legacy=0x7f2d10066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 -- 172.21.15.33:0/1693805292 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100c6740 con 0x7f2d100683c0 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.108+0000 7f2d0e59c700 1 --2- 172.21.15.33:0/1693805292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d100683c0 0x7f2d100c6f40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2d100aeba0 tx=0x7f2cf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dce6fef60f5a29c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0d59a700 1 -- 172.21.15.33:0/1693805292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8009e70 con 0x7f2d100683c0 2023-10-06T02:47:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0d59a700 1 -- 172.21.15.33:0/1693805292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf800b860 con 0x7f2d100683c0 2023-10-06T02:47:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0d59a700 1 -- 172.21.15.33:0/1693805292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800bc00 con 0x7f2d100683c0 2023-10-06T02:47:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1693805292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d100683c0 msgr2=0x7f2d100c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 --2- 172.21.15.33:0/1693805292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d100683c0 0x7f2d100c6f40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2d100aeba0 tx=0x7f2cf8009250 comp rx=0 tx=0).stop 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1693805292 shutdown_connections 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 --2- 172.21.15.33:0/1693805292 >> v2:172.21.15.167:3300/0 conn(0x7f2d10068790 0x7f2d1006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 --2- 172.21.15.33:0/1693805292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d100683c0 0x7f2d100c6f40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1693805292 >> 172.21.15.33:0/1693805292 conn(0x7f2d100cc290 msgr2=0x7f2d100c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1693805292 shutdown_connections 2023-10-06T02:47:08.118 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.112+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1693805292 wait complete. 2023-10-06T02:47:08.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 Processor -- start 2023-10-06T02:47:08.120 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 -- start start 2023-10-06T02:47:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d100683c0 0x7f2d10168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d10068790 0x7f2d100c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d100c7480 0x7f2d100c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10145240 con 0x7f2d10068790 2023-10-06T02:47:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10144630 con 0x7f2d100683c0 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10144910 con 0x7f2d100c7480 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d10068790 0x7f2d100c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d100683c0 0x7f2d10168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d10068790 0x7f2d100c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50680/0 (socket says 172.21.15.33:50680) 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0ed9d700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d100c7480 0x7f2d100c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 -- 172.21.15.33:0/1854739556 learned_addr learned my addr 172.21.15.33:0/1854739556 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 -- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d100c7480 msgr2=0x7f2d100c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d100c7480 0x7f2d100c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 -- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d100683c0 msgr2=0x7f2d10168f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d100683c0 0x7f2d10168f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 -- 172.21.15.33:0/1854739556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80094f0 con 0x7f2d10068790 2023-10-06T02:47:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0dd9b700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d10068790 0x7f2d100c1bc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000ec50 tx=0x7f2d000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000c850 con 0x7f2d10068790 2023-10-06T02:47:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000c9b0 con 0x7f2d10068790 2023-10-06T02:47:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d10144bf0 con 0x7f2d10068790 2023-10-06T02:47:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000cd70 con 0x7f2d10068790 2023-10-06T02:47:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.116+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d10169440 con 0x7f2d10068790 2023-10-06T02:47:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.120+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d10071040 con 0x7f2d10068790 2023-10-06T02:47:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.120+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2d00013680 con 0x7f2d10068790 2023-10-06T02:47:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.120+0000 7f2d067fc700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2cf0060550 0x7f2cf0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.120+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f2d0007c9c0 con 0x7f2d10068790 2023-10-06T02:47:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.120+0000 7f2d067fc700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7f2cf0066ee0 0x7f2cf0069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.120+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2cf00699e0 con 0x7f2cf0066ee0 2023-10-06T02:47:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.124+0000 7f2d0e59c700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2cf0060550 0x7f2cf0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.124+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d00081220 con 0x7f2d10068790 2023-10-06T02:47:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.124+0000 7f2d0ed9d700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7f2cf0066ee0 0x7f2cf0069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.124+0000 7f2d0ed9d700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7f2cf0066ee0 0x7f2cf0069310 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.124+0000 7f2d0e59c700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2cf0060550 0x7f2cf0062a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d100aeba0 tx=0x7f2cf8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.131 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.124+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== osd.3 v2:172.21.15.33:6817/33424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2cf00699e0 con 0x7f2cf0066ee0 2023-10-06T02:47:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 --> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2d100c6a20 con 0x7f2cf0066ee0 2023-10-06T02:47:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d067fc700 1 -- 172.21.15.33:0/1854739556 <== osd.3 v2:172.21.15.33:6817/33424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2d100c6a20 con 0x7f2cf0066ee0 2023-10-06T02:47:08.150 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:08.150 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:08.151 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7f2cf0066ee0 msgr2=0x7f2cf0069310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.154 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:6817/33424,v1:172.21.15.33:6819/33424] conn(0x7f2cf0066ee0 0x7f2cf0069310 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2cf0060550 msgr2=0x7f2cf0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f2cf0060550 0x7f2cf0062a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d100aeba0 tx=0x7f2cf8017000 comp rx=0 tx=0).stop 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d10068790 msgr2=0x7f2d100c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 --2- 172.21.15.33:0/1854739556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d10068790 0x7f2d100c1bc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000ec50 tx=0x7f2d000091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0ed9d700 1 -- 172.21.15.33:0/1854739556 reap_dead start 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 shutdown_connections 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 >> 172.21.15.33:0/1854739556 conn(0x7f2d100cc290 msgr2=0x7f2d100ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 shutdown_connections 2023-10-06T02:47:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.144+0000 7f2d0f59e700 1 -- 172.21.15.33:0/1854739556 wait complete. 2023-10-06T02:47:08.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T02:47:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 Processor -- start 2023-10-06T02:47:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 -- start start 2023-10-06T02:47:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15841584a0 0x7f15840ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15840c9a80 0x7f15840ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f15840c7a80 0x7f15840ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15840c6480 con 0x7f15840c9a80 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158406a180 con 0x7f15840c7a80 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1589752700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1584069570 con 0x7f15841584a0 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15840c9a80 0x7f15840ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15840c9a80 0x7f15840ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50696/0 (socket says 172.21.15.33:50696) 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 -- 172.21.15.33:0/1548944552 learned_addr learned my addr 172.21.15.33:0/1548944552 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1583fff700 1 --2- 172.21.15.33:0/1548944552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15841584a0 0x7f15840ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f1588f51700 1 --2- 172.21.15.33:0/1548944552 >> v2:172.21.15.167:3300/0 conn(0x7f15840c7a80 0x7f15840ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 -- 172.21.15.33:0/1548944552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15841584a0 msgr2=0x7f15840ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 --2- 172.21.15.33:0/1548944552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15841584a0 0x7f15840ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 -- 172.21.15.33:0/1548944552 >> v2:172.21.15.167:3300/0 conn(0x7f15840c7a80 msgr2=0x7f15840ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 --2- 172.21.15.33:0/1548944552 >> v2:172.21.15.167:3300/0 conn(0x7f15840c7a80 0x7f15840ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.268+0000 7f15837fe700 1 -- 172.21.15.33:0/1548944552 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584069850 con 0x7f15840c9a80 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f15837fe700 1 --2- 172.21.15.33:0/1548944552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c9a80 0x7f15840ca310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f15840c9330 tx=0x7f156c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a5e57492604d93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1582ffd700 1 -- 172.21.15.33:0/1548944552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00b640 con 0x7f15840c9a80 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1582ffd700 1 -- 172.21.15.33:0/1548944552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00b7a0 con 0x7f15840c9a80 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1582ffd700 1 -- 172.21.15.33:0/1548944552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00bb60 con 0x7f15840c9a80 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 -- 172.21.15.33:0/1548944552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c9a80 msgr2=0x7f15840ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 --2- 172.21.15.33:0/1548944552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c9a80 0x7f15840ca310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f15840c9330 tx=0x7f156c009250 comp rx=0 tx=0).stop 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 -- 172.21.15.33:0/1548944552 shutdown_connections 2023-10-06T02:47:08.279 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 --2- 172.21.15.33:0/1548944552 >> v2:172.21.15.167:3300/0 conn(0x7f15840c7a80 0x7f15840ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 --2- 172.21.15.33:0/1548944552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c9a80 0x7f15840ca310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 --2- 172.21.15.33:0/1548944552 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15841584a0 0x7f15840ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 -- 172.21.15.33:0/1548944552 >> 172.21.15.33:0/1548944552 conn(0x7f15840c8240 msgr2=0x7f15840c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 -- 172.21.15.33:0/1548944552 shutdown_connections 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.272+0000 7f1589752700 1 -- 172.21.15.33:0/1548944552 wait complete. 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 Processor -- start 2023-10-06T02:47:08.280 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 -- start start 2023-10-06T02:47:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 0x7f158414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15840c9a80 0x7f158414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15841584a0 0x7f158414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584151e50 con 0x7f15840c7a80 2023-10-06T02:47:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584152130 con 0x7f15841584a0 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1589752700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158414c4e0 con 0x7f15840c9a80 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 0x7f158414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 0x7f158414f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50698/0 (socket says 172.21.15.33:50698) 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 -- 172.21.15.33:0/3320004830 learned_addr learned my addr 172.21.15.33:0/3320004830 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f15837fe700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15840c9a80 0x7f158414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1588f51700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15841584a0 0x7f158414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 -- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15840c9a80 msgr2=0x7f158414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15840c9a80 0x7f158414e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 -- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15841584a0 msgr2=0x7f158414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15841584a0 0x7f158414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.276+0000 7f1583fff700 1 -- 172.21.15.33:0/3320004830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0094f0 con 0x7f15840c7a80 2023-10-06T02:47:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f1583fff700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 0x7f158414f1a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f157400e930 tx=0x7f15740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574013680 con 0x7f15840c7a80 2023-10-06T02:47:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15740137e0 con 0x7f15840c7a80 2023-10-06T02:47:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574013b80 con 0x7f15840c7a80 2023-10-06T02:47:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158414c7c0 con 0x7f15840c7a80 2023-10-06T02:47:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158414f900 con 0x7f15840c7a80 2023-10-06T02:47:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158414c340 con 0x7f15840c7a80 2023-10-06T02:47:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f157400c3e0 con 0x7f15840c7a80 2023-10-06T02:47:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f157bfff700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1564060550 0x7f1564062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f157407c8a0 con 0x7f15840c7a80 2023-10-06T02:47:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.280+0000 7f15837fe700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1564060550 0x7f1564062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.284+0000 7f15837fe700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1564060550 0x7f1564062a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f15840c9330 tx=0x7f156c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.284+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f158414c340 con 0x7f15840c7a80 2023-10-06T02:47:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f158414c340 con 0x7f1564060550 2023-10-06T02:47:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f157bfff700 1 -- 172.21.15.33:0/3320004830 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f158414c340 con 0x7f1564060550 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1564060550 msgr2=0x7f1564062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1564060550 0x7f1564062a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f15840c9330 tx=0x7f156c017000 comp rx=0 tx=0).stop 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 msgr2=0x7f158414f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 0x7f158414f1a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f157400e930 tx=0x7f15740091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 shutdown_connections 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1564060550 0x7f1564062a00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15840c9a80 0x7f158414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15841584a0 0x7f158414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 --2- 172.21.15.33:0/3320004830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15840c7a80 0x7f158414f1a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 >> 172.21.15.33:0/3320004830 conn(0x7f15840c8240 msgr2=0x7f15840c0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 shutdown_connections 2023-10-06T02:47:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.452+0000 7f1589752700 1 -- 172.21.15.33:0/3320004830 wait complete. 2023-10-06T02:47:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T02:47:08.476 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T02:47:08.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.592+0000 7f8a10da7700 1 Processor -- start 2023-10-06T02:47:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.592+0000 7f8a10da7700 1 -- start start 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c155840 0x7f8a0c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a0c0c7880 0x7f8a0c0af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c06a470 con 0x7f8a0c0c66e0 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0ae1c0 con 0x7f8a0c0c7880 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c069860 con 0x7f8a0c155840 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a0c0c7880 0x7f8a0c0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8a0c0c66e0 0x7f8a0c0c7170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45122/0 (socket says 172.21.15.33:45122) 2023-10-06T02:47:08.602 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a0c0c7880 0x7f8a0c0af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50700/0 (socket says 172.21.15.33:50700) 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0affd700 1 -- 172.21.15.33:0/2364022801 learned_addr learned my addr 172.21.15.33:0/2364022801 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0b7fe700 1 --2- 172.21.15.33:0/2364022801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c155840 0x7f8a0c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2364022801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c155840 msgr2=0x7f8a0c06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 --2- 172.21.15.33:0/2364022801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c155840 0x7f8a0c06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2364022801 >> v1:172.21.15.33:6789/0 conn(0x7f8a0c0c66e0 legacy=0x7f8a0c0c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2364022801 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c069b40 con 0x7f8a0c0c7880 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0bfff700 1 --2- 172.21.15.33:0/2364022801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c0c7880 0x7f8a0c0af840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009e60 tx=0x7f8a04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f65b3f55eb48c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0a7fc700 1 -- 172.21.15.33:0/2364022801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400e070 con 0x7f8a0c0c7880 2023-10-06T02:47:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0a7fc700 1 -- 172.21.15.33:0/2364022801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0400bb60 con 0x7f8a0c0c7880 2023-10-06T02:47:08.604 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a0a7fc700 1 -- 172.21.15.33:0/2364022801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04012430 con 0x7f8a0c0c7880 2023-10-06T02:47:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- 172.21.15.33:0/2364022801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c0c7880 msgr2=0x7f8a0c0af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 --2- 172.21.15.33:0/2364022801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c0c7880 0x7f8a0c0af840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009e60 tx=0x7f8a04009780 comp rx=0 tx=0).stop 2023-10-06T02:47:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- 172.21.15.33:0/2364022801 shutdown_connections 2023-10-06T02:47:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 --2- 172.21.15.33:0/2364022801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c0c7880 0x7f8a0c0af840 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 --2- 172.21.15.33:0/2364022801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c155840 0x7f8a0c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- 172.21.15.33:0/2364022801 >> 172.21.15.33:0/2364022801 conn(0x7f8a0c0cc290 msgr2=0x7f8a0c0c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- 172.21.15.33:0/2364022801 shutdown_connections 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.596+0000 7f8a10da7700 1 -- 172.21.15.33:0/2364022801 wait complete. 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 Processor -- start 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- start start 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c0c66e0 0x7f8a0c1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c0c7880 0x7f8a0c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c155840 0x7f8a0c0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c167dc0 con 0x7f8a0c155840 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1680a0 con 0x7f8a0c0c66e0 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c168380 con 0x7f8a0c0c7880 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c155840 0x7f8a0c0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c155840 0x7f8a0c0b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50702/0 (socket says 172.21.15.33:50702) 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0b7fe700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c0c66e0 0x7f8a0c1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2011703977 learned_addr learned my addr 172.21.15.33:0/2011703977 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0affd700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c0c7880 0x7f8a0c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c0c7880 msgr2=0x7f8a0c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a0c0c7880 0x7f8a0c0b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c0c66e0 msgr2=0x7f8a0c1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c0c66e0 0x7f8a0c1531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2011703977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009a20 con 0x7f8a0c155840 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0b7fe700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c0c66e0 0x7f8a0c1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a0bfff700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c155840 0x7f8a0c0b1000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a04013000 tx=0x7f8a04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04027470 con 0x7f8a0c155840 2023-10-06T02:47:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c168660 con 0x7f8a0c155840 2023-10-06T02:47:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c153710 con 0x7f8a0c155840 2023-10-06T02:47:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a040275f0 con 0x7f8a0c155840 2023-10-06T02:47:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040279b0 con 0x7f8a0c155840 2023-10-06T02:47:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.600+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89ec000fd0 con 0x7f8a0c155840 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8a04024070 con 0x7f8a0c155840 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f89f37fe700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89f8060580 0x7f89f8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f8a0407f5c0 con 0x7f8a0c155840 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f89f37fe700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f89f8066e90 0x7f89f80692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 --> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89f8069990 con 0x7f89f8066e90 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a0407f990 con 0x7f8a0c155840 2023-10-06T02:47:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f8a0b7fe700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89f8060580 0x7f89f8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f8a0affd700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f89f8066e90 0x7f89f80692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f8a0b7fe700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89f8060580 0x7f89f8062a30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8a000113e0 tx=0x7f8a000091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.604+0000 7f8a0affd700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f89f8066e90 0x7f89f80692c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.615 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.608+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== osd.4 v2:172.21.15.167:6800/33657 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f89f8069990 con 0x7f89f8066e90 2023-10-06T02:47:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.620+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 --> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f89ec002d50 con 0x7f89f8066e90 2023-10-06T02:47:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f89f37fe700 1 -- 172.21.15.33:0/2011703977 <== osd.4 v2:172.21.15.167:6800/33657 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f89ec002d50 con 0x7f89f8066e90 2023-10-06T02:47:08.628 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:08.628 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:08.628 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f89f8066e90 msgr2=0x7f89f80692c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6800/33657,v1:172.21.15.167:6801/33657] conn(0x7f89f8066e90 0x7f89f80692c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89f8060580 msgr2=0x7f89f8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f89f8060580 0x7f89f8062a30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8a000113e0 tx=0x7f8a000091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c155840 msgr2=0x7f8a0c0b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 --2- 172.21.15.33:0/2011703977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a0c155840 0x7f8a0c0b1000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a04013000 tx=0x7f8a04019000 comp rx=0 tx=0).stop 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a0bfff700 1 -- 172.21.15.33:0/2011703977 reap_dead start 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 shutdown_connections 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 >> 172.21.15.33:0/2011703977 conn(0x7f8a0c0cc290 msgr2=0x7f8a0c154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 shutdown_connections 2023-10-06T02:47:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.624+0000 7f8a10da7700 1 -- 172.21.15.33:0/2011703977 wait complete. 2023-10-06T02:47:08.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T02:47:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 Processor -- start 2023-10-06T02:47:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 -- start start 2023-10-06T02:47:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f737806d160 0x7f737806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73780cbfb0 0x7f737815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f73780cc400 0x7f737815d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73780c97e0 con 0x7f73780cbfb0 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73780ca1b0 con 0x7f73780cc400 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f737cd9c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780c8740 con 0x7f737806d160 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73780cbfb0 0x7f737815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73780cbfb0 0x7f737815cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50714/0 (socket says 172.21.15.33:50714) 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 -- 172.21.15.33:0/1232961208 learned_addr learned my addr 172.21.15.33:0/1232961208 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f73777fe700 1 --2- 172.21.15.33:0/1232961208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f737806d160 0x7f737806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7377fff700 1 --2- 172.21.15.33:0/1232961208 >> v2:172.21.15.167:3300/0 conn(0x7f73780cc400 0x7f737815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 -- 172.21.15.33:0/1232961208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f737806d160 msgr2=0x7f737806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 --2- 172.21.15.33:0/1232961208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f737806d160 0x7f737806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 -- 172.21.15.33:0/1232961208 >> v2:172.21.15.167:3300/0 conn(0x7f73780cc400 msgr2=0x7f737815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 --2- 172.21.15.33:0/1232961208 >> v2:172.21.15.167:3300/0 conn(0x7f73780cc400 0x7f737815d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 -- 172.21.15.33:0/1232961208 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780c8a20 con 0x7f73780cbfb0 2023-10-06T02:47:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f7376ffd700 1 --2- 172.21.15.33:0/1232961208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815cc20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f73780c8010 tx=0x7f7360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf99aea4a6ce0e7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f73767fc700 1 -- 172.21.15.33:0/1232961208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000b640 con 0x7f73780cbfb0 2023-10-06T02:47:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f73767fc700 1 -- 172.21.15.33:0/1232961208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736000b7a0 con 0x7f73780cbfb0 2023-10-06T02:47:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.748+0000 7f73767fc700 1 -- 172.21.15.33:0/1232961208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000bb60 con 0x7f73780cbfb0 2023-10-06T02:47:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 -- 172.21.15.33:0/1232961208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 msgr2=0x7f737815cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 --2- 172.21.15.33:0/1232961208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815cc20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f73780c8010 tx=0x7f7360009250 comp rx=0 tx=0).stop 2023-10-06T02:47:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 -- 172.21.15.33:0/1232961208 shutdown_connections 2023-10-06T02:47:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 --2- 172.21.15.33:0/1232961208 >> v2:172.21.15.167:3300/0 conn(0x7f73780cc400 0x7f737815d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 --2- 172.21.15.33:0/1232961208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815cc20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 --2- 172.21.15.33:0/1232961208 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f737806d160 0x7f737806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 -- 172.21.15.33:0/1232961208 >> 172.21.15.33:0/1232961208 conn(0x7f737806c280 msgr2=0x7f73780cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 -- 172.21.15.33:0/1232961208 shutdown_connections 2023-10-06T02:47:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.752+0000 7f737cd9c700 1 -- 172.21.15.33:0/1232961208 wait complete. 2023-10-06T02:47:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 Processor -- start 2023-10-06T02:47:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 -- start start 2023-10-06T02:47:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f737806d160 0x7f737815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780cc400 0x7f737815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737815dd00 con 0x7f73780cbfb0 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780c55c0 con 0x7f737806d160 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f737cd9c700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780c4c90 con 0x7f73780cc400 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50730/0 (socket says 172.21.15.33:50730) 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 -- 172.21.15.33:0/2820948997 learned_addr learned my addr 172.21.15.33:0/2820948997 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7377fff700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780cc400 0x7f737815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f73777fe700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f737806d160 0x7f737815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 -- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780cc400 msgr2=0x7f737815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780cc400 0x7f737815ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 -- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f737806d160 msgr2=0x7f737815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f737806d160 0x7f737815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 -- 172.21.15.33:0/2820948997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600094f0 con 0x7f73780cbfb0 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f7376ffd700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815e8a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f73600094c0 tx=0x7f7360017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.756+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360000d10 con 0x7f73780cbfb0 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736000b910 con 0x7f73780cbfb0 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73780c4f70 con 0x7f73780cbfb0 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360026430 con 0x7f73780cbfb0 2023-10-06T02:47:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737815d940 con 0x7f73780cbfb0 2023-10-06T02:47:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7378071040 con 0x7f73780cbfb0 2023-10-06T02:47:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7360023030 con 0x7f73780cbfb0 2023-10-06T02:47:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f736f7fe700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f735c060580 0x7f735c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f736007dd90 con 0x7f73780cbfb0 2023-10-06T02:47:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.760+0000 7f73777fe700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f735c060580 0x7f735c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.764+0000 7f73777fe700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f735c060580 0x7f735c062a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7368009fd0 tx=0x7f736800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:08.774 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.768+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f736004e460 con 0x7f73780cbfb0 2023-10-06T02:47:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f73780c8d00 con 0x7f735c060580 2023-10-06T02:47:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f736f7fe700 1 -- 172.21.15.33:0/2820948997 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f73780c8d00 con 0x7f735c060580 2023-10-06T02:47:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f735c060580 msgr2=0x7f735c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f735c060580 0x7f735c062a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7368009fd0 tx=0x7f736800b040 comp rx=0 tx=0).stop 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 msgr2=0x7f737815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815e8a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f73600094c0 tx=0x7f7360017000 comp rx=0 tx=0).stop 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 shutdown_connections 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f735c060580 0x7f735c062a30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73780cc400 0x7f737815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73780cbfb0 0x7f737815e8a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 --2- 172.21.15.33:0/2820948997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f737806d160 0x7f737815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 >> 172.21.15.33:0/2820948997 conn(0x7f737806c280 msgr2=0x7f7378158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 shutdown_connections 2023-10-06T02:47:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:08.928+0000 7f737cd9c700 1 -- 172.21.15.33:0/2820948997 wait complete. 2023-10-06T02:47:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T02:47:08.947 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T02:47:08.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.060+0000 7f5dc3fff700 1 Processor -- start 2023-10-06T02:47:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.060+0000 7f5dc3fff700 1 -- start start 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40cac00 0x7f5dc40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5dc40af280 0x7f5dc4068420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc4159410 con 0x7f5dc41588a0 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40cb510 con 0x7f5dc40af280 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40ce140 con 0x7f5dc40cac00 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5dc40af280 0x7f5dc4068420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc2ffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40cac00 0x7f5dc40cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5dc40af280 0x7f5dc4068420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.33:53830/0 (socket says 172.21.15.33:53830) 2023-10-06T02:47:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 -- 172.21.15.33:0/2603211873 learned_addr learned my addr 172.21.15.33:0/2603211873 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 -- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40cac00 msgr2=0x7f5dc40cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 --2- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40cac00 0x7f5dc40cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 -- 172.21.15.33:0/2603211873 >> v1:172.21.15.167:6789/0 conn(0x7f5dc41588a0 legacy=0x7f5dc4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 -- 172.21.15.33:0/2603211873 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40cd880 con 0x7f5dc40af280 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc1ffb700 1 -- 172.21.15.33:0/2603211873 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016565894 0 0) 0x7f5dbc001490 con 0x7f5dc41588a0 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc27fc700 1 --2- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc40af280 0x7f5dc4068420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5db40099b0 tx=0x7f5db4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990fd627592f170 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc1ffb700 1 -- 172.21.15.33:0/2603211873 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4009ea0 con 0x7f5dc40af280 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc1ffb700 1 -- 172.21.15.33:0/2603211873 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400b860 con 0x7f5dc40af280 2023-10-06T02:47:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc1ffb700 1 -- 172.21.15.33:0/2603211873 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400bc00 con 0x7f5dc40af280 2023-10-06T02:47:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc40af280 msgr2=0x7f5dc4068420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 --2- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc40af280 0x7f5dc4068420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5db40099b0 tx=0x7f5db4009250 comp rx=0 tx=0).stop 2023-10-06T02:47:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- 172.21.15.33:0/2603211873 shutdown_connections 2023-10-06T02:47:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 --2- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc40af280 0x7f5dc4068420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 --2- 172.21.15.33:0/2603211873 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40cac00 0x7f5dc40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- 172.21.15.33:0/2603211873 >> 172.21.15.33:0/2603211873 conn(0x7f5dc4067a30 msgr2=0x7f5dc40cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- 172.21.15.33:0/2603211873 shutdown_connections 2023-10-06T02:47:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.064+0000 7f5dc3fff700 1 -- 172.21.15.33:0/2603211873 wait complete. 2023-10-06T02:47:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 Processor -- start 2023-10-06T02:47:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 -- start start 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40af280 0x7f5dc40b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc40cac00 0x7f5dc4151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc41588a0 0x7f5dc40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b3320 con 0x7f5dc40cac00 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b57f0 con 0x7f5dc41588a0 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc3fff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40afb00 con 0x7f5dc40af280 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc40cac00 0x7f5dc4151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc40cac00 0x7f5dc4151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50746/0 (socket says 172.21.15.33:50746) 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc27fc700 1 -- 172.21.15.33:0/3711801745 learned_addr learned my addr 172.21.15.33:0/3711801745 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc2ffd700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40af280 0x7f5dc40b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.068+0000 7f5dc37fe700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc41588a0 0x7f5dc40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc27fc700 1 -- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40af280 msgr2=0x7f5dc40b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc27fc700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5dc40af280 0x7f5dc40b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc27fc700 1 -- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc41588a0 msgr2=0x7f5dc40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc27fc700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc41588a0 0x7f5dc40b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc27fc700 1 -- 172.21.15.33:0/3711801745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40094f0 con 0x7f5dc40cac00 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc37fe700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dc41588a0 0x7f5dc40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:09.077 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc27fc700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc40cac00 0x7f5dc4151150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cd6e0 tx=0x7f5db4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4000d10 con 0x7f5dc40cac00 2023-10-06T02:47:09.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400bbe0 con 0x7f5dc40cac00 2023-10-06T02:47:09.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc4151690 con 0x7f5dc40cac00 2023-10-06T02:47:09.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4011690 con 0x7f5dc40cac00 2023-10-06T02:47:09.078 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40b4f50 con 0x7f5dc40cac00 2023-10-06T02:47:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5dc4071040 con 0x7f5dc40cac00 2023-10-06T02:47:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.072+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5db4011ac0 con 0x7f5dc40cac00 2023-10-06T02:47:09.079 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dbaffd700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5da4060550 0x7f5da4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f5db407eb40 con 0x7f5dc40cac00 2023-10-06T02:47:09.080 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dbaffd700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f5da4066d70 0x7f5da40691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dc2ffd700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5da4060550 0x7f5da4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 --> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5da4069870 con 0x7f5da4066d70 2023-10-06T02:47:09.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db407ee30 con 0x7f5dc40cac00 2023-10-06T02:47:09.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dc37fe700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f5da4066d70 0x7f5da40691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.081 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dc2ffd700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5da4060550 0x7f5da4062a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009db0 tx=0x7f5dac0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.082 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dc37fe700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f5da4066d70 0x7f5da40691a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.083 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.076+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== osd.5 v2:172.21.15.167:6808/33659 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f5da4069870 con 0x7f5da4066d70 2023-10-06T02:47:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 --> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5dc40cdaf0 con 0x7f5da4066d70 2023-10-06T02:47:09.102 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dbaffd700 1 -- 172.21.15.33:0/3711801745 <== osd.5 v2:172.21.15.167:6808/33659 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5dc40cdaf0 con 0x7f5da4066d70 2023-10-06T02:47:09.102 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:09.102 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:09.102 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f5da4066d70 msgr2=0x7f5da40691a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6808/33659,v1:172.21.15.167:6809/33659] conn(0x7f5da4066d70 0x7f5da40691a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5da4060550 msgr2=0x7f5da4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f5da4060550 0x7f5da4062a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009db0 tx=0x7f5dac0091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc40cac00 msgr2=0x7f5dc4151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 --2- 172.21.15.33:0/3711801745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc40cac00 0x7f5dc4151150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cd6e0 tx=0x7f5db4017000 comp rx=0 tx=0).stop 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc37fe700 1 -- 172.21.15.33:0/3711801745 reap_dead start 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 shutdown_connections 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 >> 172.21.15.33:0/3711801745 conn(0x7f5dc4067a30 msgr2=0x7f5dc4156630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 shutdown_connections 2023-10-06T02:47:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.096+0000 7f5dc3fff700 1 -- 172.21.15.33:0/3711801745 wait complete. 2023-10-06T02:47:09.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T02:47:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 Processor -- start 2023-10-06T02:47:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- start start 2023-10-06T02:47:09.234 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc93806b050 0x7fc938066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc9380c93a0 0x7fc9380c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93806b420 con 0x7fc9380c6610 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380674f0 con 0x7fc9380c93a0 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93806a4e0 con 0x7fc93806b050 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc9380c93a0 0x7fc9380c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc9380c93a0 0x7fc9380c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50752/0 (socket says 172.21.15.33:50752) 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 -- 172.21.15.33:0/866744584 learned_addr learned my addr 172.21.15.33:0/866744584 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc9377fe700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc93806b050 0x7fc938066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.235 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 -- 172.21.15.33:0/866744584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc93806b050 msgr2=0x7fc938066fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc93806b050 0x7fc938066fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 -- 172.21.15.33:0/866744584 >> v1:172.21.15.33:6789/0 conn(0x7fc9380c6610 legacy=0x7fc9380c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 -- 172.21.15.33:0/866744584 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938067700 con 0x7fc9380c93a0 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc9377fe700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc93806b050 0x7fc938066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc936ffd700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c93a0 0x7fc9380c60d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc928009f20 tx=0x7fc928009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa00170c5000f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc9367fc700 1 -- 172.21.15.33:0/866744584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800e070 con 0x7fc9380c93a0 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc9367fc700 1 -- 172.21.15.33:0/866744584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800ba80 con 0x7fc9380c93a0 2023-10-06T02:47:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc9367fc700 1 -- 172.21.15.33:0/866744584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280092e0 con 0x7fc9380c93a0 2023-10-06T02:47:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- 172.21.15.33:0/866744584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c93a0 msgr2=0x7fc9380c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c93a0 0x7fc9380c60d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc928009f20 tx=0x7fc928009840 comp rx=0 tx=0).stop 2023-10-06T02:47:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- 172.21.15.33:0/866744584 shutdown_connections 2023-10-06T02:47:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c93a0 0x7fc9380c60d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 --2- 172.21.15.33:0/866744584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc93806b050 0x7fc938066fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- 172.21.15.33:0/866744584 >> 172.21.15.33:0/866744584 conn(0x7fc9380c8240 msgr2=0x7fc9380c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- 172.21.15.33:0/866744584 shutdown_connections 2023-10-06T02:47:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.228+0000 7fc93c814700 1 -- 172.21.15.33:0/866744584 wait complete. 2023-10-06T02:47:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 Processor -- start 2023-10-06T02:47:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 -- start start 2023-10-06T02:47:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93806b050 0x7fc93814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 0x7fc93815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9380c93a0 0x7fc93815fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93816d030 con 0x7fc9380c6610 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380b1450 con 0x7fc93806b050 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc93c814700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc938160460 con 0x7fc9380c93a0 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc936ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 0x7fc93815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc936ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 0x7fc93815f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50758/0 (socket says 172.21.15.33:50758) 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc936ffd700 1 -- 172.21.15.33:0/4077029317 learned_addr learned my addr 172.21.15.33:0/4077029317 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc9377fe700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93806b050 0x7fc93814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc937fff700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9380c93a0 0x7fc93815fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.232+0000 7fc936ffd700 1 -- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9380c93a0 msgr2=0x7fc93815fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc936ffd700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9380c93a0 0x7fc93815fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc936ffd700 1 -- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93806b050 msgr2=0x7fc93814f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc936ffd700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93806b050 0x7fc93814f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc936ffd700 1 -- 172.21.15.33:0/4077029317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928009ae0 con 0x7fc9380c6610 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc936ffd700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 0x7fc93815f8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc92800b6c0 tx=0x7fc928018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928008c60 con 0x7fc9380c6610 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928008dc0 con 0x7fc9380c6610 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc938160740 con 0x7fc9380c6610 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280266b0 con 0x7fc9380c6610 2023-10-06T02:47:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc938160d00 con 0x7fc9380c6610 2023-10-06T02:47:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc938071040 con 0x7fc9380c6610 2023-10-06T02:47:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc928003040 con 0x7fc9380c6610 2023-10-06T02:47:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc92f7fe700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc918060550 0x7fc918062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.236+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fc92807e740 con 0x7fc9380c6610 2023-10-06T02:47:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.240+0000 7fc9377fe700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc918060550 0x7fc918062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.240+0000 7fc9377fe700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc918060550 0x7fc918062a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc920006fd0 tx=0x7fc920008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.247 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.240+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc92804ed90 con 0x7fc9380c6610 2023-10-06T02:47:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.404+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9380679e0 con 0x7fc918060550 2023-10-06T02:47:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc92f7fe700 1 -- 172.21.15.33:0/4077029317 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9380679e0 con 0x7fc918060550 2023-10-06T02:47:09.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc918060550 msgr2=0x7fc918062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.414 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc918060550 0x7fc918062a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc920006fd0 tx=0x7fc920008040 comp rx=0 tx=0).stop 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 msgr2=0x7fc93815f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 0x7fc93815f8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc92800b6c0 tx=0x7fc928018000 comp rx=0 tx=0).stop 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 shutdown_connections 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fc918060550 0x7fc918062a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9380c93a0 0x7fc93815fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9380c6610 0x7fc93815f8f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 --2- 172.21.15.33:0/4077029317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93806b050 0x7fc93814f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 >> 172.21.15.33:0/4077029317 conn(0x7fc9380c8240 msgr2=0x7fc93815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 shutdown_connections 2023-10-06T02:47:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.408+0000 7fc93c814700 1 -- 172.21.15.33:0/4077029317 wait complete. 2023-10-06T02:47:09.416 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T02:47:09.426 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T02:47:09.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:09.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 Processor -- start 2023-10-06T02:47:09.544 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 -- start start 2023-10-06T02:47:09.546 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb806ad00 0x7f7bb8069a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7bb80ca790 0x7f7bb8066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80c9bf0 con 0x7f7bb80aa650 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb80cac10 con 0x7f7bb80ca790 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbe3e7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb815d9a0 con 0x7f7bb806ad00 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbc984700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7bb80ca790 0x7f7bb8066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7bb80aa650 0x7f7bb80ca080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45150/0 (socket says 172.21.15.33:45150) 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb77fe700 1 -- 172.21.15.33:0/4050967649 learned_addr learned my addr 172.21.15.33:0/4050967649 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb7fff700 1 --2- 172.21.15.33:0/4050967649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb806ad00 0x7f7bb8069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbc984700 1 -- 172.21.15.33:0/4050967649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb806ad00 msgr2=0x7f7bb8069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbc984700 1 --2- 172.21.15.33:0/4050967649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb806ad00 0x7f7bb8069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb6ffd700 1 -- 172.21.15.33:0/4050967649 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281185474 0 0) 0x7f7bb80c9bf0 con 0x7f7bb80aa650 2023-10-06T02:47:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbc984700 1 -- 172.21.15.33:0/4050967649 >> v1:172.21.15.33:6789/0 conn(0x7f7bb80aa650 legacy=0x7f7bb80ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbc984700 1 -- 172.21.15.33:0/4050967649 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80688e0 con 0x7f7bb80ca790 2023-10-06T02:47:09.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bbc984700 1 --2- 172.21.15.33:0/4050967649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80ca790 0x7f7bb8066fb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009e60 tx=0x7f7bb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b24f804e3b1c2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.549 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb6ffd700 1 -- 172.21.15.33:0/4050967649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000e070 con 0x7f7bb80ca790 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb6ffd700 1 -- 172.21.15.33:0/4050967649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb000bb60 con 0x7f7bb80ca790 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.540+0000 7f7bb6ffd700 1 -- 172.21.15.33:0/4050967649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0012430 con 0x7f7bb80ca790 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/4050967649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80ca790 msgr2=0x7f7bb8066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 --2- 172.21.15.33:0/4050967649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80ca790 0x7f7bb8066fb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009e60 tx=0x7f7bb0009780 comp rx=0 tx=0).stop 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/4050967649 shutdown_connections 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 --2- 172.21.15.33:0/4050967649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80ca790 0x7f7bb8066fb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 --2- 172.21.15.33:0/4050967649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb806ad00 0x7f7bb8069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/4050967649 >> 172.21.15.33:0/4050967649 conn(0x7f7bb80cc560 msgr2=0x7f7bb80c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/4050967649 shutdown_connections 2023-10-06T02:47:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/4050967649 wait complete. 2023-10-06T02:47:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 Processor -- start 2023-10-06T02:47:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- start start 2023-10-06T02:47:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb806ad00 0x7f7bb814af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80aa650 0x7f7bb815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb80ca790 0x7f7bb815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80af890 con 0x7f7bb80aa650 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80afb70 con 0x7f7bb806ad00 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bbe3e7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80afe50 con 0x7f7bb80ca790 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80aa650 0x7f7bb815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.544+0000 7f7bb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80aa650 0x7f7bb815ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50772/0 (socket says 172.21.15.33:50772) 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 -- 172.21.15.33:0/2079080320 learned_addr learned my addr 172.21.15.33:0/2079080320 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb7fff700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb806ad00 0x7f7bb814af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bbc984700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb80ca790 0x7f7bb815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 -- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb80ca790 msgr2=0x7f7bb815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb80ca790 0x7f7bb815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 -- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb806ad00 msgr2=0x7f7bb814af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb806ad00 0x7f7bb814af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 -- 172.21.15.33:0/2079080320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0009a20 con 0x7f7bb80aa650 2023-10-06T02:47:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bbc984700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bb80ca790 0x7f7bb815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T02:47:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bb77fe700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80aa650 0x7f7bb815ec70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00c5b0 tx=0x7f7bac001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac005b30 con 0x7f7bb80aa650 2023-10-06T02:47:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac005c90 con 0x7f7bb80aa650 2023-10-06T02:47:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb80b0130 con 0x7f7bb80aa650 2023-10-06T02:47:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac007300 con 0x7f7bb80aa650 2023-10-06T02:47:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb80aeb80 con 0x7f7bb80aa650 2023-10-06T02:47:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.548+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7bb80afce0 con 0x7f7bb80aa650 2023-10-06T02:47:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7bac01b3b0 con 0x7f7bb80aa650 2023-10-06T02:47:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7b9ffff700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7ba4060550 0x7f7ba4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f7bac07bca0 con 0x7f7bb80aa650 2023-10-06T02:47:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7b9ffff700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7f7ba4066ee0 0x7f7ba4069310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ba40699e0 con 0x7f7ba4066ee0 2023-10-06T02:47:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bac07e2e0 con 0x7f7bb80aa650 2023-10-06T02:47:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7bb7fff700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7ba4060550 0x7f7ba4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7bbc984700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7f7ba4066ee0 0x7f7ba4069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7bb7fff700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7ba4060550 0x7f7ba4062a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009ca0 tx=0x7f7ba8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.552+0000 7f7bbc984700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7f7ba4066ee0 0x7f7ba4069310 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.556+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== osd.6 v2:172.21.15.167:6816/33656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7ba40699e0 con 0x7f7ba4066ee0 2023-10-06T02:47:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.572+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 --> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7bb80704c0 con 0x7f7ba4066ee0 2023-10-06T02:47:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.572+0000 7f7b9ffff700 1 -- 172.21.15.33:0/2079080320 <== osd.6 v2:172.21.15.167:6816/33656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7bb80704c0 con 0x7f7ba4066ee0 2023-10-06T02:47:09.579 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:09.580 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7f7ba4066ee0 msgr2=0x7f7ba4069310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6816/33656,v1:172.21.15.167:6817/33656] conn(0x7f7ba4066ee0 0x7f7ba4069310 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7ba4060550 msgr2=0x7f7ba4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f7ba4060550 0x7f7ba4062a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009ca0 tx=0x7f7ba8008040 comp rx=0 tx=0).stop 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80aa650 msgr2=0x7f7bb815ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 --2- 172.21.15.33:0/2079080320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bb80aa650 0x7f7bb815ec70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00c5b0 tx=0x7f7bac001070 comp rx=0 tx=0).stop 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbc984700 1 -- 172.21.15.33:0/2079080320 reap_dead start 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 shutdown_connections 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 >> 172.21.15.33:0/2079080320 conn(0x7f7bb80cc560 msgr2=0x7f7bb80c2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 shutdown_connections 2023-10-06T02:47:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.576+0000 7f7bbe3e7700 1 -- 172.21.15.33:0/2079080320 wait complete. 2023-10-06T02:47:09.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T02:47:09.710 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280ffff700 1 Processor -- start 2023-10-06T02:47:09.711 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280ffff700 1 -- start start 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f28100c7420 0x7f2810159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280ffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100688b0 con 0x7f2810159b60 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280ffff700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2810067ca0 con 0x7f2810159250 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280ffff700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2810067f80 con 0x7f28100c7420 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2810159b60 0x7f2810162810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45160/0 (socket says 172.21.15.33:45160) 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.704+0000 7f280f7fe700 1 -- 172.21.15.33:0/2064713579 learned_addr learned my addr 172.21.15.33:0/2064713579 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280e7fc700 1 --2- 172.21.15.33:0/2064713579 >> v2:172.21.15.167:3300/0 conn(0x7f28100c7420 0x7f2810159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796975302 0 0) 0x7f28100688b0 con 0x7f2810159b60 2023-10-06T02:47:09.712 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27fc003570 con 0x7f2810159b60 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204838094 0 0) 0x7f2810067ca0 con 0x7f2810159250 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28100688b0 con 0x7f2810159250 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4175951066 0 0) 0x7f27fc003570 con 0x7f2810159b60 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 >> v2:172.21.15.167:3300/0 conn(0x7f28100c7420 msgr2=0x7f2810159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 --2- 172.21.15.33:0/2064713579 >> v2:172.21.15.167:3300/0 conn(0x7f28100c7420 0x7f2810159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 >> v1:172.21.15.167:6789/0 conn(0x7f2810159250 legacy=0x7f28100c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.717 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810068260 con 0x7f2810159b60 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2299766788 0 0) 0x7f2808001470 con 0x7f2810159b60 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 >> v1:172.21.15.33:6789/0 conn(0x7f2810159b60 legacy=0x7f2810162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc003bf0 0x7f27fc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc004810 0x7f27fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc0051c0 0x7f27fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810067f80 con 0x7f27fc003bf0 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc00c0b0 con 0x7f27fc0051c0 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc00c270 con 0x7f27fc004810 2023-10-06T02:47:09.718 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280e7fc700 1 --2- 172.21.15.33:0/2064713579 >> v2:172.21.15.167:3300/0 conn(0x7f28100c7420 0x7f2810159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc003bf0 0x7f27fc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280f7fe700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc0051c0 0x7f27fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280e7fc700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc004810 0x7f27fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 -- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc004810 msgr2=0x7f27fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc004810 0x7f27fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 -- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc0051c0 msgr2=0x7f27fc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc0051c0 0x7f27fc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 -- 172.21.15.33:0/2064713579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc003570 con 0x7f27fc003bf0 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280f7fe700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27fc0051c0 0x7f27fc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280f7fe700 1 -- 172.21.15.33:0/2064713579 reap_dead start 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280effd700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc003bf0 0x7f27fc003fe0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f27f8001a20 tx=0x7f27f8007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e12e1bc1b1193f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.719 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80013b0 con 0x7f27fc003bf0 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f800c5d0 con 0x7f27fc003bf0 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280dffb700 1 -- 172.21.15.33:0/2064713579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800c990 con 0x7f27fc003bf0 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280ffff700 1 -- 172.21.15.33:0/2064713579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc003bf0 msgr2=0x7f27fc003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.708+0000 7f280ffff700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc003bf0 0x7f27fc003fe0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f27f8001a20 tx=0x7f27f8007cf0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- 172.21.15.33:0/2064713579 shutdown_connections 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 --2- 172.21.15.33:0/2064713579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27fc003bf0 0x7f27fc003fe0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- 172.21.15.33:0/2064713579 >> 172.21.15.33:0/2064713579 conn(0x7f28100c8240 msgr2=0x7f28100c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- 172.21.15.33:0/2064713579 shutdown_connections 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- 172.21.15.33:0/2064713579 wait complete. 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 Processor -- start 2023-10-06T02:47:09.720 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- start start 2023-10-06T02:47:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc003bf0 0x7f28101504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 0x7f281015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2810152030 0x7f2810152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281015d6b0 con 0x7f281015c2c0 2023-10-06T02:47:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28101529c0 con 0x7f2810152030 2023-10-06T02:47:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281015ce10 con 0x7f27fc003bf0 2023-10-06T02:47:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 0x7f281015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 0x7f281015c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50802/0 (socket says 172.21.15.33:50802) 2023-10-06T02:47:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 -- 172.21.15.33:0/3938354496 learned_addr learned my addr 172.21.15.33:0/3938354496 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280f7fe700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2810152030 0x7f2810152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280effd700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc003bf0 0x7f28101504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 -- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc003bf0 msgr2=0x7f28101504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc003bf0 0x7f28101504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 -- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2810152030 msgr2=0x7f2810152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2810152030 0x7f2810152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 -- 172.21.15.33:0/3938354496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281015d0f0 con 0x7f281015c2c0 2023-10-06T02:47:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280e7fc700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 0x7f281015c730 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f280000f310 tx=0x7f280000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000e040 con 0x7f281015c2c0 2023-10-06T02:47:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280000f040 con 0x7f281015c2c0 2023-10-06T02:47:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281015d3d0 con 0x7f281015c2c0 2023-10-06T02:47:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800015680 con 0x7f281015c2c0 2023-10-06T02:47:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.712+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2810150a00 con 0x7f281015c2c0 2023-10-06T02:47:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.716+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810071040 con 0x7f281015c2c0 2023-10-06T02:47:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.716+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2800015a30 con 0x7f281015c2c0 2023-10-06T02:47:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.716+0000 7f2806ffd700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f27f0060550 0x7f27f0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.716+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f280007d730 con 0x7f281015c2c0 2023-10-06T02:47:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.716+0000 7f280effd700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f27f0060550 0x7f27f0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:09.729 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.720+0000 7f280effd700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f27f0060550 0x7f27f0062a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:09.730 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.724+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f280004dda0 con 0x7f281015c2c0 2023-10-06T02:47:09.897 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.892+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2810068540 con 0x7f27f0060550 2023-10-06T02:47:09.898 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.892+0000 7f2806ffd700 1 -- 172.21.15.33:0/3938354496 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2810068540 con 0x7f27f0060550 2023-10-06T02:47:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f27f0060550 msgr2=0x7f27f0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f27f0060550 0x7f27f0062a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f8015000 comp rx=0 tx=0).stop 2023-10-06T02:47:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 msgr2=0x7f281015c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 0x7f281015c730 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f280000f310 tx=0x7f280000b040 comp rx=0 tx=0).stop 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 shutdown_connections 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2810152030 0x7f2810152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f281015c2c0 0x7f281015c730 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f27f0060550 0x7f27f0062a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 --2- 172.21.15.33:0/3938354496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27fc003bf0 0x7f28101504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 >> 172.21.15.33:0/3938354496 conn(0x7f28100c8240 msgr2=0x7f28101513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 shutdown_connections 2023-10-06T02:47:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:09.896+0000 7f280ffff700 1 -- 172.21.15.33:0/3938354496 wait complete. 2023-10-06T02:47:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T02:47:09.913 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T02:47:09.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T02:47:10.026 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.020+0000 7f1dad398700 1 Processor -- start 2023-10-06T02:47:10.027 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.020+0000 7f1dad398700 1 -- start start 2023-10-06T02:47:10.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cca80 0x7f1da80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1da815c5b0 0x7f1da80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da80cf430 con 0x7f1da80cf8c0 2023-10-06T02:47:10.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da80cfd10 con 0x7f1da815c5b0 2023-10-06T02:47:10.028 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80ce820 con 0x7f1da80cca80 2023-10-06T02:47:10.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1d9e7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1da80cf8c0 0x7f1da815bea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45174/0 (socket says 172.21.15.33:45174) 2023-10-06T02:47:10.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1d9e7fc700 1 -- 172.21.15.33:0/2150560504 learned_addr learned my addr 172.21.15.33:0/2150560504 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:10.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da6ffd700 1 --2- 172.21.15.33:0/2150560504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cca80 0x7f1da80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.029 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da77fe700 1 --2- 172.21.15.33:0/2150560504 >> v2:172.21.15.167:3300/0 conn(0x7f1da815c5b0 0x7f1da80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da67fc700 1 -- 172.21.15.33:0/2150560504 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666763379 0 0) 0x7f1da80cf430 con 0x7f1da80cf8c0 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da67fc700 1 -- 172.21.15.33:0/2150560504 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d94003570 con 0x7f1da80cf8c0 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da6ffd700 1 -- 172.21.15.33:0/2150560504 >> v2:172.21.15.167:3300/0 conn(0x7f1da815c5b0 msgr2=0x7f1da80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da6ffd700 1 --2- 172.21.15.33:0/2150560504 >> v2:172.21.15.167:3300/0 conn(0x7f1da815c5b0 0x7f1da80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da6ffd700 1 -- 172.21.15.33:0/2150560504 >> v1:172.21.15.33:6789/0 conn(0x7f1da80cf8c0 legacy=0x7f1da815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da6ffd700 1 -- 172.21.15.33:0/2150560504 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80ceb00 con 0x7f1da80cca80 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da6ffd700 1 --2- 172.21.15.33:0/2150560504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cca80 0x7f1da80cce50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d980098e0 tx=0x7f1d980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c910ca91b93e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da67fc700 1 -- 172.21.15.33:0/2150560504 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98011630 con 0x7f1da80cca80 2023-10-06T02:47:10.030 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da67fc700 1 -- 172.21.15.33:0/2150560504 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98011790 con 0x7f1da80cca80 2023-10-06T02:47:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1da67fc700 1 -- 172.21.15.33:0/2150560504 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98011b10 con 0x7f1da80cca80 2023-10-06T02:47:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- 172.21.15.33:0/2150560504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cca80 msgr2=0x7f1da80cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 --2- 172.21.15.33:0/2150560504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cca80 0x7f1da80cce50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d980098e0 tx=0x7f1d980091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- 172.21.15.33:0/2150560504 shutdown_connections 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 --2- 172.21.15.33:0/2150560504 >> v2:172.21.15.167:3300/0 conn(0x7f1da815c5b0 0x7f1da80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 --2- 172.21.15.33:0/2150560504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cca80 0x7f1da80cce50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- 172.21.15.33:0/2150560504 >> 172.21.15.33:0/2150560504 conn(0x7f1da8067a40 msgr2=0x7f1da8068690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- 172.21.15.33:0/2150560504 shutdown_connections 2023-10-06T02:47:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.024+0000 7f1dad398700 1 -- 172.21.15.33:0/2150560504 wait complete. 2023-10-06T02:47:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 Processor -- start 2023-10-06T02:47:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 -- start start 2023-10-06T02:47:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da80cca80 0x7f1da80cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cf8c0 0x7f1da80c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da815c5b0 0x7f1da80c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80c9030 con 0x7f1da815c5b0 2023-10-06T02:47:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815e4b0 con 0x7f1da80cca80 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1dad398700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80c9230 con 0x7f1da80cf8c0 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da815c5b0 0x7f1da80c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da815c5b0 0x7f1da80c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50818/0 (socket says 172.21.15.33:50818) 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 -- 172.21.15.33:0/2080438839 learned_addr learned my addr 172.21.15.33:0/2080438839 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da6ffd700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da80cca80 0x7f1da80cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1d9e7fc700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cf8c0 0x7f1da80c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 -- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cf8c0 msgr2=0x7f1da80c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cf8c0 0x7f1da80c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 -- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da80cca80 msgr2=0x7f1da80cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da80cca80 0x7f1da80cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 -- 172.21.15.33:0/2080438839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d98009420 con 0x7f1da815c5b0 2023-10-06T02:47:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1d9e7fc700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1da80cf8c0 0x7f1da80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:47:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.028+0000 7f1da77fe700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da815c5b0 0x7f1da80c8af0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009df0 tx=0x7f1da000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da000cb50 con 0x7f1da815c5b0 2023-10-06T02:47:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000f040 con 0x7f1da815c5b0 2023-10-06T02:47:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da815db80 con 0x7f1da815c5b0 2023-10-06T02:47:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0014480 con 0x7f1da815c5b0 2023-10-06T02:47:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da815e140 con 0x7f1da815c5b0 2023-10-06T02:47:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1da80704c0 con 0x7f1da815c5b0 2023-10-06T02:47:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1da0014830 con 0x7f1da815c5b0 2023-10-06T02:47:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1d9f7fe700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1d88060580 0x7f1d88062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.032+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f1da007d030 con 0x7f1da815c5b0 2023-10-06T02:47:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1da6ffd700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1d88060580 0x7f1d88062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1d9f7fe700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7f1d88066f10 0x7f1d88069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 --> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d88069a10 con 0x7f1d88066f10 2023-10-06T02:47:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1da007f670 con 0x7f1da815c5b0 2023-10-06T02:47:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1d9e7fc700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7f1d88066f10 0x7f1d88069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1da6ffd700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1d88060580 0x7f1d88062a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1d980091e0 tx=0x7f1d98017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1d9e7fc700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7f1d88066f10 0x7f1d88069340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.042 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.036+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== osd.7 v2:172.21.15.167:6824/33658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1d88069a10 con 0x7f1d88066f10 2023-10-06T02:47:10.059 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.052+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 --> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1da80cede0 con 0x7f1d88066f10 2023-10-06T02:47:10.060 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1d9f7fe700 1 -- 172.21.15.33:0/2080438839 <== osd.7 v2:172.21.15.167:6824/33658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1da80cede0 con 0x7f1d88066f10 2023-10-06T02:47:10.060 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2023-10-06T02:47:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T02:47:10.061 INFO:teuthology.orchestra.run.smithi033.stdout:} 2023-10-06T02:47:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7f1d88066f10 msgr2=0x7f1d88069340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6824/33658,v1:172.21.15.167:6825/33658] conn(0x7f1d88066f10 0x7f1d88069340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1d88060580 msgr2=0x7f1d88062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f1d88060580 0x7f1d88062a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1d980091e0 tx=0x7f1d98017000 comp rx=0 tx=0).stop 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da815c5b0 msgr2=0x7f1da80c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 --2- 172.21.15.33:0/2080438839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da815c5b0 0x7f1da80c8af0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009df0 tx=0x7f1da000b040 comp rx=0 tx=0).stop 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1da77fe700 1 -- 172.21.15.33:0/2080438839 reap_dead start 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 shutdown_connections 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 >> 172.21.15.33:0/2080438839 conn(0x7f1da8067a40 msgr2=0x7f1da80687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 shutdown_connections 2023-10-06T02:47:10.062 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.056+0000 7f1dad398700 1 -- 172.21.15.33:0/2080438839 wait complete. 2023-10-06T02:47:10.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T02:47:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.180+0000 7fa510083700 1 Processor -- start 2023-10-06T02:47:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 -- start start 2023-10-06T02:47:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 0x7fa5080aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5080679c0 0x7fa508067e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa508157650 0x7fa508157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5080c6c10 con 0x7fa5080679c0 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa508068350 con 0x7fa508157650 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa510083700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa508157ff0 con 0x7fa508157280 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50d61e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5080679c0 0x7fa508067e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 0x7fa5080aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 0x7fa5080aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:45456/0 (socket says 172.21.15.33:45456) 2023-10-06T02:47:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 -- 172.21.15.33:0/341720907 learned_addr learned my addr 172.21.15.33:0/341720907 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50e620700 1 --2- 172.21.15.33:0/341720907 >> v2:172.21.15.167:3300/0 conn(0x7fa508157650 0x7fa508157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 -- 172.21.15.33:0/341720907 >> v2:172.21.15.167:3300/0 conn(0x7fa508157650 msgr2=0x7fa508157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 --2- 172.21.15.33:0/341720907 >> v2:172.21.15.167:3300/0 conn(0x7fa508157650 0x7fa508157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 -- 172.21.15.33:0/341720907 >> v2:172.21.15.33:3300/0 conn(0x7fa5080679c0 msgr2=0x7fa508067e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 --2- 172.21.15.33:0/341720907 >> v2:172.21.15.33:3300/0 conn(0x7fa5080679c0 0x7fa508067e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 -- 172.21.15.33:0/341720907 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080685d0 con 0x7fa508157280 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50d61e700 1 --2- 172.21.15.33:0/341720907 >> v2:172.21.15.33:3300/0 conn(0x7fa5080679c0 0x7fa508067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50de1f700 1 --2- 172.21.15.33:0/341720907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 0x7fa5080aadd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa5080c7ae0 tx=0x7fa500009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11ebf594ed193c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50ce1d700 1 -- 172.21.15.33:0/341720907 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000e070 con 0x7fa508157280 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50ce1d700 1 -- 172.21.15.33:0/341720907 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50000b970 con 0x7fa508157280 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.184+0000 7fa50ce1d700 1 -- 172.21.15.33:0/341720907 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000092e0 con 0x7fa508157280 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 -- 172.21.15.33:0/341720907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 msgr2=0x7fa5080aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 --2- 172.21.15.33:0/341720907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 0x7fa5080aadd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa5080c7ae0 tx=0x7fa500009840 comp rx=0 tx=0).stop 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 -- 172.21.15.33:0/341720907 shutdown_connections 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 --2- 172.21.15.33:0/341720907 >> v2:172.21.15.167:3300/0 conn(0x7fa508157650 0x7fa508157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 --2- 172.21.15.33:0/341720907 >> v2:172.21.15.33:3300/0 conn(0x7fa5080679c0 0x7fa508067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 --2- 172.21.15.33:0/341720907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157280 0x7fa5080aadd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 -- 172.21.15.33:0/341720907 >> 172.21.15.33:0/341720907 conn(0x7fa5080c83f0 msgr2=0x7fa5080c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 -- 172.21.15.33:0/341720907 shutdown_connections 2023-10-06T02:47:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 -- 172.21.15.33:0/341720907 wait complete. 2023-10-06T02:47:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 Processor -- start 2023-10-06T02:47:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.188+0000 7fa510083700 1 -- start start 2023-10-06T02:47:10.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 0x7fa508166ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa508157280 0x7fa5080aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.196 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157650 0x7fa5080b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508158570 con 0x7fa5080679c0 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508159c90 con 0x7fa508157280 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa508159f70 con 0x7fa508157650 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 0x7fa508166ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 0x7fa508166ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50838/0 (socket says 172.21.15.33:50838) 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 -- 172.21.15.33:0/2601987398 learned_addr learned my addr 172.21.15.33:0/2601987398 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50e620700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157650 0x7fa5080b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50d61e700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa508157280 0x7fa5080aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 -- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157650 msgr2=0x7fa5080b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157650 0x7fa5080b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 -- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa508157280 msgr2=0x7fa5080aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.197 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa508157280 0x7fa5080aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 -- 172.21.15.33:0/2601987398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc009420 con 0x7fa5080679c0 2023-10-06T02:47:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa50de1f700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 0x7fa508166ac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa50000b610 tx=0x7fa500018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500008a00 con 0x7fa5080679c0 2023-10-06T02:47:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa500008b60 con 0x7fa5080679c0 2023-10-06T02:47:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa500009ae0 con 0x7fa5080679c0 2023-10-06T02:47:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500026460 con 0x7fa5080679c0 2023-10-06T02:47:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa508158200 con 0x7fa5080679c0 2023-10-06T02:47:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5080710b0 con 0x7fa5080679c0 2023-10-06T02:47:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.192+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa500003040 con 0x7fa5080679c0 2023-10-06T02:47:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.196+0000 7fa505ffb700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa4f4060550 0x7fa4f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.196+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7fa50007e4d0 con 0x7fa5080679c0 2023-10-06T02:47:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.196+0000 7fa50d61e700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa4f4060550 0x7fa4f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.196+0000 7fa50d61e700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa4f4060550 0x7fa4f4062a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0093f0 tx=0x7fa4fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.200+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa50004eb20 con 0x7fa5080679c0 2023-10-06T02:47:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.372+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5081586b0 con 0x7fa4f4060550 2023-10-06T02:47:10.376 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.372+0000 7fa505ffb700 1 -- 172.21.15.33:0/2601987398 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa4fc0119c0 con 0x7fa4f4060550 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa4f4060550 msgr2=0x7fa4f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa4f4060550 0x7fa4f4062a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0093f0 tx=0x7fa4fc008040 comp rx=0 tx=0).stop 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 msgr2=0x7fa508166ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 0x7fa508166ac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa50000b610 tx=0x7fa500018000 comp rx=0 tx=0).stop 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 shutdown_connections 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7fa4f4060550 0x7fa4f4062a00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa508157650 0x7fa5080b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa508157280 0x7fa5080aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 --2- 172.21.15.33:0/2601987398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5080679c0 0x7fa508166ac0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 >> 172.21.15.33:0/2601987398 conn(0x7fa5080c83f0 msgr2=0x7fa508069150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 shutdown_connections 2023-10-06T02:47:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.376+0000 7fa510083700 1 -- 172.21.15.33:0/2601987398 wait complete. 2023-10-06T02:47:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T02:47:10.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:47:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 Processor -- start 2023-10-06T02:47:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- start start 2023-10-06T02:47:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7f415d600 0x7ff7f406bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff7f40b06d0 0x7ff7f406c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f40af6c0 con 0x7ff7f4158bc0 2023-10-06T02:47:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40af910 con 0x7ff7f415d600 2023-10-06T02:47:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40afbf0 con 0x7ff7f40b06d0 2023-10-06T02:47:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7f415d600 0x7ff7f406bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7f415d600 0x7ff7f406bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50854/0 (socket says 172.21.15.33:50854) 2023-10-06T02:47:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 -- 172.21.15.33:0/4194466523 learned_addr learned my addr 172.21.15.33:0/4194466523 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f2d9d700 1 --2- 172.21.15.33:0/4194466523 >> v2:172.21.15.167:3300/0 conn(0x7ff7f40b06d0 0x7ff7f406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 -- 172.21.15.33:0/4194466523 >> v2:172.21.15.167:3300/0 conn(0x7ff7f40b06d0 msgr2=0x7ff7f406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 --2- 172.21.15.33:0/4194466523 >> v2:172.21.15.167:3300/0 conn(0x7ff7f40b06d0 0x7ff7f406c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 -- 172.21.15.33:0/4194466523 >> v1:172.21.15.167:6789/0 conn(0x7ff7f4158bc0 legacy=0x7ff7f40b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 -- 172.21.15.33:0/4194466523 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f40afed0 con 0x7ff7f415d600 2023-10-06T02:47:10.507 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7ebfff700 1 --2- 172.21.15.33:0/4194466523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f415d600 0x7ff7f406bde0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406ca40 tx=0x7ff7dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e858a08e83d4508 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f1d9b700 1 -- 172.21.15.33:0/4194466523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc009e70 con 0x7ff7f415d600 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f1d9b700 1 -- 172.21.15.33:0/4194466523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc00b860 con 0x7ff7f415d600 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f1d9b700 1 -- 172.21.15.33:0/4194466523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00bc00 con 0x7ff7f415d600 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- 172.21.15.33:0/4194466523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f415d600 msgr2=0x7ff7f406bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 --2- 172.21.15.33:0/4194466523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f415d600 0x7ff7f406bde0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406ca40 tx=0x7ff7dc009250 comp rx=0 tx=0).stop 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- 172.21.15.33:0/4194466523 shutdown_connections 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 --2- 172.21.15.33:0/4194466523 >> v2:172.21.15.167:3300/0 conn(0x7ff7f40b06d0 0x7ff7f406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 --2- 172.21.15.33:0/4194466523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f415d600 0x7ff7f406bde0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- 172.21.15.33:0/4194466523 >> 172.21.15.33:0/4194466523 conn(0x7ff7f406e6c0 msgr2=0x7ff7f406ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- 172.21.15.33:0/4194466523 shutdown_connections 2023-10-06T02:47:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.500+0000 7ff7f359e700 1 -- 172.21.15.33:0/4194466523 wait complete. 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 Processor -- start 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 -- start start 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f40b06d0 0x7ff7f40a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 0x7ff7f41401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7f4156740 0x7ff7f4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4157830 con 0x7ff7f4158bc0 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4156c20 con 0x7ff7f40b06d0 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4156f00 con 0x7ff7f4156740 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 0x7ff7f41401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 0x7ff7f41401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50856/0 (socket says 172.21.15.33:50856) 2023-10-06T02:47:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 -- 172.21.15.33:0/1037943500 learned_addr learned my addr 172.21.15.33:0/1037943500 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f259c700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f40b06d0 0x7ff7f40a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f2d9d700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7f4156740 0x7ff7f4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 -- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7f4156740 msgr2=0x7ff7f4157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7f4156740 0x7ff7f4157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 -- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f40b06d0 msgr2=0x7ff7f40a88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f40b06d0 0x7ff7f40a88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 -- 172.21.15.33:0/1037943500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc0094f0 con 0x7ff7f4158bc0 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ebfff700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 0x7ff7f41401a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406bae0 tx=0x7ff7dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00bbe0 con 0x7ff7f4158bc0 2023-10-06T02:47:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc000c90 con 0x7ff7f4158bc0 2023-10-06T02:47:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f41571e0 con 0x7ff7f4158bc0 2023-10-06T02:47:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc011650 con 0x7ff7f4158bc0 2023-10-06T02:47:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.504+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f414e900 con 0x7ff7f4158bc0 2023-10-06T02:47:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.508+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff7dc011a40 con 0x7ff7f4158bc0 2023-10-06T02:47:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.508+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f4157660 con 0x7ff7f4158bc0 2023-10-06T02:47:10.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.508+0000 7ff7ea7fc700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff7d4060550 0x7ff7d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:10.514 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.508+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7ff7dc07dca0 con 0x7ff7f4158bc0 2023-10-06T02:47:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.508+0000 7ff7f259c700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff7d4060550 0x7ff7d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.508+0000 7ff7f259c700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff7d4060550 0x7ff7d4062a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e4007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.512+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff7dc04e250 con 0x7ff7f4158bc0 2023-10-06T02:47:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.672+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7f4071040 con 0x7ff7d4060550 2023-10-06T02:47:10.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.680+0000 7ff7ea7fc700 1 -- 172.21.15.33:0/1037943500 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70930 (secure 0 0 0) 0x7ff7f4071040 con 0x7ff7d4060550 2023-10-06T02:47:10.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff7d4060550 msgr2=0x7ff7d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff7d4060550 0x7ff7d4062a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e4007e70 comp rx=0 tx=0).stop 2023-10-06T02:47:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 msgr2=0x7ff7f41401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 0x7ff7f41401a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406bae0 tx=0x7ff7dc017000 comp rx=0 tx=0).stop 2023-10-06T02:47:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 shutdown_connections 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7ff7d4060550 0x7ff7d4062a00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff7f4156740 0x7ff7f4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f4158bc0 0x7ff7f41401a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 --2- 172.21.15.33:0/1037943500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f40b06d0 0x7ff7f40a88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 >> 172.21.15.33:0/1037943500 conn(0x7ff7f406e6c0 msgr2=0x7ff7f406bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 shutdown_connections 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:10.684+0000 7ff7f359e700 1 -- 172.21.15.33:0/1037943500 wait complete. 2023-10-06T02:47:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:47:10.707 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-10-06T02:47:10.597171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2391976,"kb_used_data":5480,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752582744,"statfs":{"total":773094113280,"available":770644729856,"internally_reserved":0,"allocated":5611520,"data_stored":3154152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.167456"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:47:01.437605+0000","last_change":"2023-10-06T02:45:18.647557+0000","last_active":"2023-10-06T02:47:01.437605+0000","last_peered":"2023-10-06T02:47:01.437605+0000","last_clean":"2023-10-06T02:47:01.437605+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:47:01.437605+0000","last_undegraded":"2023-10-06T02:47:01.437605+0000","last_fullsized":"2023-10-06T02:47:01.437605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:52:47.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:02.440156+0000","last_change":"2023-10-06T02:45:18.642618+0000","last_active":"2023-10-06T02:47:02.440156+0000","last_peered":"2023-10-06T02:47:02.440156+0000","last_clean":"2023-10-06T02:47:02.440156+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:47:02.440156+0000","last_undegraded":"2023-10-06T02:47:02.440156+0000","last_fullsized":"2023-10-06T02:47:02.440156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:47:01.648106+0000","last_change":"2023-10-06T02:45:18.646793+0000","last_active":"2023-10-06T02:47:01.648106+0000","last_peered":"2023-10-06T02:47:01.648106+0000","last_clean":"2023-10-06T02:47:01.648106+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:47:01.648106+0000","last_undegraded":"2023-10-06T02:47:01.648106+0000","last_fullsized":"2023-10-06T02:47:01.648106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:06.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:02.439867+0000","last_change":"2023-10-06T02:45:18.642985+0000","last_active":"2023-10-06T02:47:02.439867+0000","last_peered":"2023-10-06T02:47:02.439867+0000","last_clean":"2023-10-06T02:47:02.439867+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:47:02.439867+0000","last_undegraded":"2023-10-06T02:47:02.439867+0000","last_fullsized":"2023-10-06T02:47:02.439867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:02.437024+0000","last_change":"2023-10-06T02:45:18.649224+0000","last_active":"2023-10-06T02:47:02.437024+0000","last_peered":"2023-10-06T02:47:02.437024+0000","last_clean":"2023-10-06T02:47:02.437024+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:47:02.437024+0000","last_undegraded":"2023-10-06T02:47:02.437024+0000","last_fullsized":"2023-10-06T02:47:02.437024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:02.606547+0000","last_change":"2023-10-06T02:45:18.671276+0000","last_active":"2023-10-06T02:47:02.606547+0000","last_peered":"2023-10-06T02:47:02.606547+0000","last_clean":"2023-10-06T02:47:02.606547+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:47:02.606547+0000","last_undegraded":"2023-10-06T02:47:02.606547+0000","last_fullsized":"2023-10-06T02:47:02.606547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:47:01.643373+0000","last_change":"2023-10-06T02:45:18.649268+0000","last_active":"2023-10-06T02:47:01.643373+0000","last_peered":"2023-10-06T02:47:01.643373+0000","last_clean":"2023-10-06T02:47:01.643373+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:47:01.643373+0000","last_undegraded":"2023-10-06T02:47:01.643373+0000","last_fullsized":"2023-10-06T02:47:01.643373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:03.775004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":32,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:02.438591+0000","last_change":"2023-10-06T02:45:18.646618+0000","last_active":"2023-10-06T02:47:02.438591+0000","last_peered":"2023-10-06T02:47:02.438591+0000","last_clean":"2023-10-06T02:47:02.438591+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:47:02.438591+0000","last_undegraded":"2023-10-06T02:47:02.438591+0000","last_fullsized":"2023-10-06T02:47:02.438591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:17.605474+0000","last_clean_scrub_stamp":"2023-10-06T02:45:17.605474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":184,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T02:47:01.643613+0000","last_change":"2023-10-06T02:45:12.623079+0000","last_active":"2023-10-06T02:47:01.643613+0000","last_peered":"2023-10-06T02:47:01.643613+0000","last_clean":"2023-10-06T02:47:01.643613+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:47:01.643613+0000","last_undegraded":"2023-10-06T02:47:01.643613+0000","last_fullsized":"2023-10-06T02:47:01.643613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:45:11.548157+0000","last_clean_scrub_stamp":"2023-10-06T02:45:11.548157+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:53.886484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":1146880,"data_stored":837045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.79700000000000004}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.069},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.84599999999999997}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.141}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.03},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.196}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.075}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.88800000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.1240000000000001}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.2290000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.123},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.753},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.0029999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.875},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":1.242}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.21}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":3.2909999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.181},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.306}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.94499999999999995}]},{"osd":1,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.99399999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.121}]},{"osd":6,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.0920000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:46:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.21},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.97099999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.85699999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.751}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83399999999999996}]},{"osd":1,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.628}]},{"osd":3,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.98499999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.367}]},{"osd":6,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.522},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.4059999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.47}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.62}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.155}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.0249999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.0649999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":1146880,"data_stored":837045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.879},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.101}]},{"osd":5,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.97599999999999998}]},{"osd":6,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.1799999999999999}]},{"osd":7,"last update":"Fri Oct 6 02:46:13 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.081},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":1.2170000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:47:10.712 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.712 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.712 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.712 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.712 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.713 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.713 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.713 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T02:45:17.605474+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=17, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.713 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T02:45:11.548157+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=45, tm_sec=11, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=2, tm_min=47, tm_sec=5, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T02:47:10.713 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T02:47:19.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:47:19.964 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:47:30.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T02:47:30.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 Processor -- start 2023-10-06T02:47:30.824 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 -- start start 2023-10-06T02:47:30.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2510155d30 0x7f25100680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:30.825 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510069580 0x7f2510159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2510156fe0 con 0x7f2510156100 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251006a970 con 0x7f2510155d30 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2518519700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251006a380 con 0x7f2510069580 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2515ab4700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2510156100 0x7f2510068600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42846/0 (socket says 172.21.15.33:42846) 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2515ab4700 1 -- 172.21.15.33:0/2099847095 learned_addr learned my addr 172.21.15.33:0/2099847095 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25162b5700 1 --2- 172.21.15.33:0/2099847095 >> v2:172.21.15.167:3300/0 conn(0x7f2510155d30 0x7f25100680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:30.826 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f2516ab6700 1 --2- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510069580 0x7f2510159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:30.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25152b3700 1 -- 172.21.15.33:0/2099847095 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302076878 0 0) 0x7f2510156fe0 con 0x7f2510156100 2023-10-06T02:47:30.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25152b3700 1 -- 172.21.15.33:0/2099847095 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24fc003570 con 0x7f2510156100 2023-10-06T02:47:30.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25162b5700 1 -- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510069580 msgr2=0x7f2510159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:30.827 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25162b5700 1 --2- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510069580 0x7f2510159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:30.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25162b5700 1 -- 172.21.15.33:0/2099847095 >> v1:172.21.15.33:6789/0 conn(0x7f2510156100 legacy=0x7f2510068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:30.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25162b5700 1 -- 172.21.15.33:0/2099847095 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25101572c0 con 0x7f2510155d30 2023-10-06T02:47:30.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25162b5700 1 --2- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510155d30 0x7f25100680c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f250c0098d0 tx=0x7f250c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e7d65239b3a496 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:30.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25152b3700 1 -- 172.21.15.33:0/2099847095 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c005c10 con 0x7f2510155d30 2023-10-06T02:47:30.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25152b3700 1 -- 172.21.15.33:0/2099847095 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c005d70 con 0x7f2510155d30 2023-10-06T02:47:30.828 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.820+0000 7f25152b3700 1 -- 172.21.15.33:0/2099847095 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c006260 con 0x7f2510155d30 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510155d30 msgr2=0x7f25100680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 --2- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510155d30 0x7f25100680c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f250c0098d0 tx=0x7f250c009120 comp rx=0 tx=0).stop 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/2099847095 shutdown_connections 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 --2- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510069580 0x7f2510159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 --2- 172.21.15.33:0/2099847095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510155d30 0x7f25100680c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/2099847095 >> 172.21.15.33:0/2099847095 conn(0x7f25100af920 msgr2=0x7f25100afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:30.829 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/2099847095 shutdown_connections 2023-10-06T02:47:30.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/2099847095 wait complete. 2023-10-06T02:47:30.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 Processor -- start 2023-10-06T02:47:30.830 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- start start 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 0x7f2510169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510155d30 0x7f251016c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510156100 0x7f2510067c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100c34c0 con 0x7f2510069580 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100c1950 con 0x7f2510156100 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251016ce70 con 0x7f2510155d30 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 0x7f2510169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 0x7f2510169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45908/0 (socket says 172.21.15.33:45908) 2023-10-06T02:47:30.832 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 -- 172.21.15.33:0/1097285943 learned_addr learned my addr 172.21.15.33:0/1097285943 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2515ab4700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510155d30 0x7f251016c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2516ab6700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510156100 0x7f2510067c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 -- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510155d30 msgr2=0x7f251016c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510155d30 0x7f251016c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 -- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510156100 msgr2=0x7f2510067c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510156100 0x7f2510067c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 -- 172.21.15.33:0/1097285943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c0093c0 con 0x7f2510069580 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2515ab4700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510155d30 0x7f251016c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25162b5700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 0x7f2510169d30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f250c009ec0 tx=0x7f250c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c01d9c0 con 0x7f2510069580 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c01db20 con 0x7f2510069580 2023-10-06T02:47:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251016d150 con 0x7f2510069580 2023-10-06T02:47:30.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c01b480 con 0x7f2510069580 2023-10-06T02:47:30.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.824+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251016c5c0 con 0x7f2510069580 2023-10-06T02:47:30.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.828+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f250c01b830 con 0x7f2510069580 2023-10-06T02:47:30.834 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.828+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25100c1b50 con 0x7f2510069580 2023-10-06T02:47:30.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.828+0000 7f25027fc700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f24f4060550 0x7f24f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:30.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.828+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f250c07e780 con 0x7f2510069580 2023-10-06T02:47:30.835 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.828+0000 7f2515ab4700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f24f4060550 0x7f24f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:30.836 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.832+0000 7f2515ab4700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f24f4060550 0x7f24f4062a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f251015aa00 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:30.843 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.836+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f25100c1b50 con 0x7f2510069580 2023-10-06T02:47:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:30.996+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 --> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25100c17d0 con 0x7f24f4060550 2023-10-06T02:47:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.004+0000 7f25027fc700 1 -- 172.21.15.33:0/1097285943 <== mgr.4100 v2:172.21.15.167:6832/33311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70930 (secure 0 0 0) 0x7f2504005900 con 0x7f24f4060550 2023-10-06T02:47:31.012 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:31.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f24f4060550 msgr2=0x7f24f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f24f4060550 0x7f24f4062a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f251015aa00 tx=0x7f2504009040 comp rx=0 tx=0).stop 2023-10-06T02:47:31.013 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 msgr2=0x7f2510169d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 0x7f2510169d30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f250c009ec0 tx=0x7f250c016000 comp rx=0 tx=0).stop 2023-10-06T02:47:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 shutdown_connections 2023-10-06T02:47:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f24f4060550 0x7f24f4062a00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2510155d30 0x7f251016c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2510156100 0x7f2510067c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.014 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 --2- 172.21.15.33:0/1097285943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510069580 0x7f2510169d30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 >> 172.21.15.33:0/1097285943 conn(0x7f25100af920 msgr2=0x7f251015b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:31.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 shutdown_connections 2023-10-06T02:47:31.015 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.008+0000 7f2518519700 1 -- 172.21.15.33:0/1097285943 wait complete. 2023-10-06T02:47:31.015 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2023-10-06T02:47:31.026 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-10-06T02:47:30.602827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2391976,"kb_used_data":5480,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752582744,"statfs":{"total":773094113280,"available":770644729856,"internally_reserved":0,"allocated":5611520,"data_stored":3154152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003343"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:11.087443+0000","last_change":"2023-10-06T02:47:11.087382+0000","last_active":"2023-10-06T02:47:11.087443+0000","last_peered":"2023-10-06T02:47:11.087443+0000","last_clean":"2023-10-06T02:47:11.087443+0000","last_became_active":"2023-10-06T02:45:18.647064+0000","last_became_peered":"2023-10-06T02:45:18.647064+0000","last_unstale":"2023-10-06T02:47:11.087443+0000","last_undegraded":"2023-10-06T02:47:11.087443+0000","last_fullsized":"2023-10-06T02:47:11.087443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:11.087131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:11.087131+0000","last_clean_scrub_stamp":"2023-10-06T02:47:11.087131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:01.738031+0000","scrub_duration":0.0041772570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:10.488504+0000","last_change":"2023-10-06T02:47:10.488466+0000","last_active":"2023-10-06T02:47:10.488504+0000","last_peered":"2023-10-06T02:47:10.488504+0000","last_clean":"2023-10-06T02:47:10.488504+0000","last_became_active":"2023-10-06T02:45:18.642008+0000","last_became_peered":"2023-10-06T02:45:18.642008+0000","last_unstale":"2023-10-06T02:47:10.488504+0000","last_undegraded":"2023-10-06T02:47:10.488504+0000","last_fullsized":"2023-10-06T02:47:10.488504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:10.488315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:10.488315+0000","last_clean_scrub_stamp":"2023-10-06T02:47:10.488315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:19.462443+0000","scrub_duration":0.0039486319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:14.506949+0000","last_change":"2023-10-06T02:47:14.506884+0000","last_active":"2023-10-06T02:47:14.506949+0000","last_peered":"2023-10-06T02:47:14.506949+0000","last_clean":"2023-10-06T02:47:14.506949+0000","last_became_active":"2023-10-06T02:45:18.646313+0000","last_became_peered":"2023-10-06T02:45:18.646313+0000","last_unstale":"2023-10-06T02:47:14.506949+0000","last_undegraded":"2023-10-06T02:47:14.506949+0000","last_fullsized":"2023-10-06T02:47:14.506949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:14.506648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:14.506648+0000","last_clean_scrub_stamp":"2023-10-06T02:47:14.506648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:20.180098+0000","scrub_duration":0.0046117570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:13.601440+0000","last_change":"2023-10-06T02:47:13.601373+0000","last_active":"2023-10-06T02:47:13.601440+0000","last_peered":"2023-10-06T02:47:13.601440+0000","last_clean":"2023-10-06T02:47:13.601440+0000","last_became_active":"2023-10-06T02:45:18.642551+0000","last_became_peered":"2023-10-06T02:45:18.642551+0000","last_unstale":"2023-10-06T02:47:13.601440+0000","last_undegraded":"2023-10-06T02:47:13.601440+0000","last_fullsized":"2023-10-06T02:47:13.601440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:13.601127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:13.601127+0000","last_clean_scrub_stamp":"2023-10-06T02:47:13.601127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T02:55:55.505498+0000","scrub_duration":0.0045921130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:10.186250+0000","last_change":"2023-10-06T02:47:10.186132+0000","last_active":"2023-10-06T02:47:10.186250+0000","last_peered":"2023-10-06T02:47:10.186250+0000","last_clean":"2023-10-06T02:47:10.186250+0000","last_became_active":"2023-10-06T02:45:18.648760+0000","last_became_peered":"2023-10-06T02:45:18.648760+0000","last_unstale":"2023-10-06T02:47:10.186250+0000","last_undegraded":"2023-10-06T02:47:10.186250+0000","last_fullsized":"2023-10-06T02:47:10.186250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:10.185895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:10.185895+0000","last_clean_scrub_stamp":"2023-10-06T02:47:10.185895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:56.941248+0000","scrub_duration":0.0043650090000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:09.846866+0000","last_change":"2023-10-06T02:47:09.846755+0000","last_active":"2023-10-06T02:47:09.846866+0000","last_peered":"2023-10-06T02:47:09.846866+0000","last_clean":"2023-10-06T02:47:09.846866+0000","last_became_active":"2023-10-06T02:45:18.670586+0000","last_became_peered":"2023-10-06T02:45:18.670586+0000","last_unstale":"2023-10-06T02:47:09.846866+0000","last_undegraded":"2023-10-06T02:47:09.846866+0000","last_fullsized":"2023-10-06T02:47:09.846866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:09.846516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:09.846516+0000","last_clean_scrub_stamp":"2023-10-06T02:47:09.846516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:13.984338+0000","scrub_duration":0.0039532500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:13.505350+0000","last_change":"2023-10-06T02:47:13.505280+0000","last_active":"2023-10-06T02:47:13.505350+0000","last_peered":"2023-10-06T02:47:13.505350+0000","last_clean":"2023-10-06T02:47:13.505350+0000","last_became_active":"2023-10-06T02:45:18.648764+0000","last_became_peered":"2023-10-06T02:45:18.648764+0000","last_unstale":"2023-10-06T02:47:13.505350+0000","last_undegraded":"2023-10-06T02:47:13.505350+0000","last_fullsized":"2023-10-06T02:47:13.505350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:13.505021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:13.505021+0000","last_clean_scrub_stamp":"2023-10-06T02:47:13.505021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:15.040648+0000","scrub_duration":0.0051476109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:11.143377+0000","last_change":"2023-10-06T02:47:11.143310+0000","last_active":"2023-10-06T02:47:11.143377+0000","last_peered":"2023-10-06T02:47:11.143377+0000","last_clean":"2023-10-06T02:47:11.143377+0000","last_became_active":"2023-10-06T02:45:18.646177+0000","last_became_peered":"2023-10-06T02:45:18.646177+0000","last_unstale":"2023-10-06T02:47:11.143377+0000","last_undegraded":"2023-10-06T02:47:11.143377+0000","last_fullsized":"2023-10-06T02:47:11.143377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T02:47:11.143079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T02:47:11.143079+0000","last_clean_scrub_stamp":"2023-10-06T02:47:11.143079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:12.678707+0000","scrub_duration":0.0040750539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"47'119","reported_seq":197,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-06T02:47:12.515660+0000","last_change":"2023-10-06T02:47:12.515584+0000","last_active":"2023-10-06T02:47:12.515660+0000","last_peered":"2023-10-06T02:47:12.515660+0000","last_clean":"2023-10-06T02:47:12.515660+0000","last_became_active":"2023-10-06T02:45:12.621065+0000","last_became_peered":"2023-10-06T02:45:12.621065+0000","last_unstale":"2023-10-06T02:47:12.515660+0000","last_undegraded":"2023-10-06T02:47:12.515660+0000","last_fullsized":"2023-10-06T02:47:12.515660+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'119","last_scrub_stamp":"2023-10-06T02:47:12.515386+0000","last_deep_scrub":"47'119","last_deep_scrub_stamp":"2023-10-06T02:47:12.515386+0000","last_clean_scrub_stamp":"2023-10-06T02:47:12.515386+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:59.270739+0000","scrub_duration":0.013267197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":1146880,"data_stored":837045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.91000000000000003,"15min":0.90700000000000003},"min":{"1min":0.58199999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.1379999999999999,"5min":1.256,"15min":1.256},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1579999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.2130000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.032,"15min":1.0469999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.407,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.87}]},{"osd":2,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64700000000000002,"5min":0.623,"15min":0.623},"max":{"1min":1.1870000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95299999999999996},"min":{"1min":0.629,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3140000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.0720000000000001}]},{"osd":4,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.98199999999999998,"15min":0.98699999999999999},"min":{"1min":0.80700000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.141,"5min":1.421,"15min":1.421},"last":1.097},{"interface":"front","average":{"1min":1.105,"5min":1.089,"15min":1.087},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.355,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.048,"15min":1.054},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.29,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1000000000000001,"15min":1.0900000000000001},"min":{"1min":0.93500000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.302,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Oct 6 02:47:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.169,"15min":1.161},"min":{"1min":1.1060000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.359,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.165},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":1.0409999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.448,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.1379999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.95099999999999996,"15min":0.96399999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3180000000000001,"5min":1.329,"15min":1.329},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.006,"15min":1.006},"min":{"1min":0.67100000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2549999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.94199999999999995,"15min":0.94899999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1080000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.87,"15min":0.88500000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.196,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.94399999999999995}]},{"osd":2,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.075},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95799999999999996,"15min":0.96399999999999997},"min":{"1min":0.64500000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1930000000000001,"5min":1.278,"15min":1.278},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.91400000000000003,"15min":0.90200000000000002},"min":{"1min":0.745,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.752},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0680000000000001,"15min":1.0669999999999999},"min":{"1min":0.76100000000000001,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.1439999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93400000000000005,"15min":0.93799999999999994},"min":{"1min":0.627,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1339999999999999,"15min":1.127},"min":{"1min":0.875,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.482,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.05,"15min":1.048},"min":{"1min":0.75700000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.111,"15min":1.117},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.014}]},{"osd":7,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.85399999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6120000000000001,"5min":1.831,"15min":1.831},"last":1.103},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2170000000000001,"15min":1.2190000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.2350000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2490000000000001,"5min":1.843,"15min":1.843},"last":1.181},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.97599999999999998,"15min":0.99399999999999999},"min":{"1min":0.58399999999999996,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.242,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.208}]},{"osd":1,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.999,"15min":1.012},"min":{"1min":0.65600000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.2350000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.98399999999999999,"15min":1.0009999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.24,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94399999999999995,"15min":0.94199999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.272,"5min":1.571,"15min":1.571},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.097,"15min":1.107},"min":{"1min":0.81100000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.3,"5min":2.298,"15min":2.298},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0309999999999999,"15min":1.036},"min":{"1min":0.77100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.411,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2230000000000001,"15min":1.2490000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.3660000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.90700000000000003}]},{"osd":4,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.63400000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3420000000000001,"5min":2.722,"15min":2.722},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.1699999999999999,"15min":1.1719999999999999},"min":{"1min":0.89100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.575,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.141}]},{"osd":6,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.125,"15min":1.1180000000000001},"min":{"1min":0.82099999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4910000000000001,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.875},{"interface":"front","average":{"1min":1.135,"5min":1.2390000000000001,"15min":1.256},"min":{"1min":0.80000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.349,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.2310000000000001}]},{"osd":7,"last update":"Fri Oct 6 02:47:16 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6659999999999999,"5min":3.2909999999999999,"15min":3.2909999999999999},"last":1.091},{"interface":"front","average":{"1min":1.27,"5min":1.4019999999999999,"15min":1.4239999999999999},"min":{"1min":1.002,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.5649999999999999,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.0580000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2010000000000001,"5min":1.282,"15min":1.282},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.087,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1899999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.92200000000000004,"15min":0.93799999999999994},"min":{"1min":0.63300000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.165,"5min":1.349,"15min":1.349},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92400000000000004,"15min":0.92200000000000004},"min":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.292,"5min":1.573,"15min":1.573},"last":0.872},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.127,"15min":1.139},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2529999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.0649999999999999}]},{"osd":3,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2070000000000001,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.02,"15min":1.024},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2350000000000001,"5min":1.831,"15min":1.831},"last":0.89200000000000002}]},{"osd":5,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.01,"15min":1.018},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2829999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.3640000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.1000000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0569999999999999,"15min":1.0669999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3340000000000001,"5min":1.413,"15min":1.413},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2150000000000001,"15min":1.2190000000000001},"min":{"1min":1.024,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.393,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.96699999999999997}]},{"osd":7,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.177,"15min":1.1759999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.216,"15min":1.228},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.3520000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.94599999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94899999999999995,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.075}]},{"osd":2,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.874,"15min":0.88200000000000001},"min":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.97199999999999998,"15min":0.98199999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.123,"5min":1.615,"15min":1.615},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.006,"15min":1.008},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3939999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.878},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.137,"15min":1.149},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.411,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.01}]},{"osd":5,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.169,"15min":1.1950000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3740000000000001,"5min":1.458,"15min":1.458},"last":1.117},{"interface":"front","average":{"1min":1.117,"5min":1.145,"15min":1.149},"min":{"1min":0.82999999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.3200000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.113,"15min":1.1240000000000001},"min":{"1min":0.81599999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.339,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.23,"15min":1.2450000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.375,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.175}]},{"osd":7,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.244,"15min":1.248},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.663,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.751,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.1439999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.79500000000000004,"15min":0.78200000000000003},"min":{"1min":0.66300000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.0640000000000001}]},{"osd":1,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.54200000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.77200000000000002,"15min":0.76100000000000001},"min":{"1min":0.628,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.874,"15min":0.86799999999999999},"min":{"1min":0.60399999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3400000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.0209999999999999}]},{"osd":4,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.028,"15min":1.0189999999999999},"min":{"1min":0.80500000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.4490000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.159},{"interface":"front","average":{"1min":1.159,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.198}]},{"osd":5,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.123,"15min":1.117},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.355},{"interface":"front","average":{"1min":1.234,"5min":1.2190000000000001,"15min":1.2170000000000001},"min":{"1min":0.95899999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.6839999999999999,"5min":1.744,"15min":1.744},"last":1.1080000000000001}]},{"osd":6,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1639999999999999,"15min":1.1579999999999999},"min":{"1min":0.85899999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1599999999999999,"15min":1.155},"min":{"1min":0.91600000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.502,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.256}]},{"osd":7,"last update":"Fri Oct 6 02:47:12 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.1799999999999999,"15min":1.1719999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.256,"15min":1.244},"min":{"1min":0.995,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.234}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":246677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.57599999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0720000000000001,"5min":1.073,"15min":1.073},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.55000000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.92300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.747}]},{"osd":2,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2430000000000001,"5min":1.444,"15min":1.444},"last":0.90300000000000002}]},{"osd":3,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92900000000000005,"15min":0.93700000000000006},"min":{"1min":0.55600000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0800000000000001,"5min":1.931,"15min":1.931},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.83999999999999997},"min":{"1min":0.52400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.143,"5min":1.325,"15min":1.325},"last":0.94899999999999995}]},{"osd":4,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0289999999999999,"15min":1.0349999999999999},"min":{"1min":0.77900000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1890000000000001,"5min":1.413,"15min":1.413},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.747,"5min":0.624,"15min":0.624},"max":{"1min":1.3360000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3979999999999999,"5min":1.498,"15min":1.498},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.73899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.347,"5min":1.762,"15min":1.762},"last":1.5229999999999999}]},{"osd":6,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.125,"15min":1.1299999999999999},"min":{"1min":0.86599999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.456,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":0.88400000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.4530000000000001,"5min":1.982,"15min":1.982},"last":0.97899999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2290000000000001,"15min":1.2370000000000001},"min":{"1min":0.86599999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.409,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.579},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1870000000000001,"15min":1.1879999999999999},"min":{"1min":0.93999999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.55,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1299999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":1146880,"data_stored":837045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88300000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93300000000000005,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.83999999999999997},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.885,"15min":1.885},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.83799999999999997,"15min":0.84799999999999998},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0640000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.72399999999999998},"min":{"1min":0.51900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.95499999999999996,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.95899999999999996,"15min":0.96499999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.296,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.91600000000000004}]},{"osd":4,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":1.0049999999999999,"15min":1.014},"min":{"1min":0.71499999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1899999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.004},{"interface":"front","average":{"1min":1.107,"5min":1.1759999999999999,"15min":1.1879999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5169999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.107}]},{"osd":5,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.66600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":1.2450000000000001,"5min":2.359,"15min":2.359},"last":1.141},{"interface":"front","average":{"1min":1.091,"5min":1.121,"15min":1.1259999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.546,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.88}]},{"osd":6,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.264,"15min":1.276},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.677,"5min":2.411,"15min":2.411},"last":1.018},{"interface":"front","average":{"1min":1.125,"5min":1.1930000000000001,"15min":1.204},"min":{"1min":0.72799999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.585,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.97499999999999998}]},{"osd":7,"last update":"Fri Oct 6 02:47:13 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2130000000000001,"15min":1.2270000000000001},"min":{"1min":0.79700000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4790000000000001,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2050000000000001,"15min":1.2110000000000001},"min":{"1min":0.88900000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.427,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":0.95499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T02:47:31.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T02:47:31.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.128+0000 7f0b04e74700 1 Processor -- start 2023-10-06T02:47:31.135 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.128+0000 7f0b04e74700 1 -- start start 2023-10-06T02:47:31.137 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0b000ac140 0x7f0b000c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b00069ba0 con 0x7f0b00068ad0 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0006f530 con 0x7f0b000ac140 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00068f90 con 0x7f0b000ac510 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afdd9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0b00068ad0 0x7f0b000aa7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42860/0 (socket says 172.21.15.33:42860) 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afdd9b700 1 -- 172.21.15.33:0/3245837280 learned_addr learned my addr 172.21.15.33:0/3245837280 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afed9d700 1 --2- 172.21.15.33:0/3245837280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afe59c700 1 --2- 172.21.15.33:0/3245837280 >> v2:172.21.15.167:3300/0 conn(0x7f0b000ac140 0x7f0b000c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.138 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afd59a700 1 -- 172.21.15.33:0/3245837280 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673912015 0 0) 0x7f0b00069ba0 con 0x7f0b00068ad0 2023-10-06T02:47:31.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afd59a700 1 -- 172.21.15.33:0/3245837280 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aec003570 con 0x7f0b00068ad0 2023-10-06T02:47:31.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afed9d700 1 -- 172.21.15.33:0/3245837280 >> v2:172.21.15.167:3300/0 conn(0x7f0b000ac140 msgr2=0x7f0b000c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:31.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afed9d700 1 --2- 172.21.15.33:0/3245837280 >> v2:172.21.15.167:3300/0 conn(0x7f0b000ac140 0x7f0b000c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afed9d700 1 -- 172.21.15.33:0/3245837280 >> v1:172.21.15.33:6789/0 conn(0x7f0b00068ad0 legacy=0x7f0b000aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.139 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afed9d700 1 -- 172.21.15.33:0/3245837280 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00069270 con 0x7f0b000ac510 2023-10-06T02:47:31.140 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afed9d700 1 --2- 172.21.15.33:0/3245837280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000c38f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009e60 tx=0x7f0af8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c08068af5ab32588 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afd59a700 1 -- 172.21.15.33:0/3245837280 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af800e070 con 0x7f0b000ac510 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afd59a700 1 -- 172.21.15.33:0/3245837280 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af800bab0 con 0x7f0b000ac510 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0afd59a700 1 -- 172.21.15.33:0/3245837280 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af800be70 con 0x7f0b000ac510 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 -- 172.21.15.33:0/3245837280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 msgr2=0x7f0b000c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.132+0000 7f0b04e74700 1 --2- 172.21.15.33:0/3245837280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000c38f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009e60 tx=0x7f0af8009780 comp rx=0 tx=0).stop 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 -- 172.21.15.33:0/3245837280 shutdown_connections 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 --2- 172.21.15.33:0/3245837280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000c38f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 --2- 172.21.15.33:0/3245837280 >> v2:172.21.15.167:3300/0 conn(0x7f0b000ac140 0x7f0b000c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 -- 172.21.15.33:0/3245837280 >> 172.21.15.33:0/3245837280 conn(0x7f0b00096ef0 msgr2=0x7f0b0006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 -- 172.21.15.33:0/3245837280 shutdown_connections 2023-10-06T02:47:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 -- 172.21.15.33:0/3245837280 wait complete. 2023-10-06T02:47:31.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 Processor -- start 2023-10-06T02:47:31.143 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.136+0000 7f0b04e74700 1 -- start start 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 0x7f0b00158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b000ac140 0x7f0b000aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0015cd60 con 0x7f0b00068ad0 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0015ca10 con 0x7f0b000ac140 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000adc70 con 0x7f0b000ac510 2023-10-06T02:47:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 0x7f0b00158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 0x7f0b00158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45912/0 (socket says 172.21.15.33:45912) 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 -- 172.21.15.33:0/886847294 learned_addr learned my addr 172.21.15.33:0/886847294 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afed9d700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afdd9b700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b000ac140 0x7f0b000aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 -- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 msgr2=0x7f0b000af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.148 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 -- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b000ac140 msgr2=0x7f0b000aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b000ac140 0x7f0b000aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 -- 172.21.15.33:0/886847294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af8009a20 con 0x7f0b00068ad0 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0afe59c700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 0x7f0b00158e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0af000ec80 tx=0x7f0af00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af000c850 con 0x7f0b00068ad0 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af000c9b0 con 0x7f0b00068ad0 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b000adf50 con 0x7f0b00068ad0 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af000cd50 con 0x7f0b00068ad0 2023-10-06T02:47:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.140+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b00098680 con 0x7f0b00068ad0 2023-10-06T02:47:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.144+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b000710b0 con 0x7f0b00068ad0 2023-10-06T02:47:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.144+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0af0013680 con 0x7f0b00068ad0 2023-10-06T02:47:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.144+0000 7f0af67fc700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0ae0060580 0x7f0ae0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.144+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f0af007d200 con 0x7f0b00068ad0 2023-10-06T02:47:31.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.144+0000 7f0afdd9b700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0ae0060580 0x7f0ae0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.151 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.144+0000 7f0afdd9b700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0ae0060580 0x7f0ae0062a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0b0006d990 tx=0x7f0ae8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:31.153 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.148+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0af004d7b0 con 0x7f0b00068ad0 2023-10-06T02:47:31.319 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.312+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0b000ab960 con 0x7f0b00068ad0 2023-10-06T02:47:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0af004d7b0 con 0x7f0b00068ad0 2023-10-06T02:47:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0af67fc700 1 -- 172.21.15.33:0/886847294 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0af0050dd0 con 0x7f0b00068ad0 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0ae0060580 msgr2=0x7f0ae0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0ae0060580 0x7f0ae0062a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0b0006d990 tx=0x7f0ae8009040 comp rx=0 tx=0).stop 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 msgr2=0x7f0b00158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 0x7f0b00158e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0af000ec80 tx=0x7f0af00091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 shutdown_connections 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f0ae0060580 0x7f0ae0062a30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b000ac510 0x7f0b000af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b000ac140 0x7f0b000aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 --2- 172.21.15.33:0/886847294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b00068ad0 0x7f0b00158e10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.324+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 >> 172.21.15.33:0/886847294 conn(0x7f0b00096ef0 msgr2=0x7f0b000cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:31.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.328+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 shutdown_connections 2023-10-06T02:47:31.333 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.328+0000 7f0b04e74700 1 -- 172.21.15.33:0/886847294 wait complete. 2023-10-06T02:47:31.346 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T02:47:31.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T02:47:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 Processor -- start 2023-10-06T02:47:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 -- start start 2023-10-06T02:47:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f50c80ade00 0x7f50c80c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80c5940 0x7f50c80c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c806add0 con 0x7f50c8066fb0 2023-10-06T02:47:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80c80c0 con 0x7f50c80ade00 2023-10-06T02:47:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cefd1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80c7c40 con 0x7f50c80c5940 2023-10-06T02:47:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50ce7d0700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f50c8066fb0 0x7f50c80c9550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:42862/0 (socket says 172.21.15.33:42862) 2023-10-06T02:47:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50ce7d0700 1 -- 172.21.15.33:0/1129845789 learned_addr learned my addr 172.21.15.33:0/1129845789 (peer_addr_for_me v1:172.21.15.33:0/0) 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cd7ce700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80c5940 0x7f50c80c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.452+0000 7f50cdfcf700 1 --2- 172.21.15.33:0/1129845789 >> v2:172.21.15.167:3300/0 conn(0x7f50c80ade00 0x7f50c80c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50ccfcd700 1 -- 172.21.15.33:0/1129845789 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302885814 0 0) 0x7f50c806add0 con 0x7f50c8066fb0 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50ccfcd700 1 -- 172.21.15.33:0/1129845789 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b4003570 con 0x7f50c8066fb0 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cdfcf700 1 -- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80c5940 msgr2=0x7f50c80c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cdfcf700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80c5940 0x7f50c80c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cdfcf700 1 -- 172.21.15.33:0/1129845789 >> v1:172.21.15.33:6789/0 conn(0x7f50c8066fb0 legacy=0x7f50c80c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cdfcf700 1 -- 172.21.15.33:0/1129845789 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80ae1d0 con 0x7f50c80ade00 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cd7ce700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80c5940 0x7f50c80c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cdfcf700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c80c71c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f50c40098e0 tx=0x7f50c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56554c639ac05e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50ccfcd700 1 -- 172.21.15.33:0/1129845789 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4011630 con 0x7f50c80ade00 2023-10-06T02:47:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50ccfcd700 1 -- 172.21.15.33:0/1129845789 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f50c4011790 con 0x7f50c80ade00 2023-10-06T02:47:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50ccfcd700 1 -- 172.21.15.33:0/1129845789 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4011b10 con 0x7f50c80ade00 2023-10-06T02:47:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 -- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 msgr2=0x7f50c80c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.463 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c80c71c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f50c40098e0 tx=0x7f50c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 -- 172.21.15.33:0/1129845789 shutdown_connections 2023-10-06T02:47:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c80c5940 0x7f50c80c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 --2- 172.21.15.33:0/1129845789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c80c71c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 -- 172.21.15.33:0/1129845789 >> 172.21.15.33:0/1129845789 conn(0x7f50c80c8240 msgr2=0x7f50c80c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 -- 172.21.15.33:0/1129845789 shutdown_connections 2023-10-06T02:47:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.456+0000 7f50cefd1700 1 -- 172.21.15.33:0/1129845789 wait complete. 2023-10-06T02:47:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 Processor -- start 2023-10-06T02:47:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 -- start start 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 0x7f50c8067840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c8153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 0x7f50c814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80686c0 con 0x7f50c80c5940 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8153700 con 0x7f50c80ade00 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50cefd1700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c814b8e0 con 0x7f50c8066fb0 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.460+0000 7f50ce7d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 0x7f50c814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cdfcf700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 0x7f50c8067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cdfcf700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 0x7f50c8067840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.33:46050/0 (socket says 172.21.15.33:46050) 2023-10-06T02:47:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cd7ce700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c8153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 0x7f50c814ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45926/0 (socket says 172.21.15.33:45926) 2023-10-06T02:47:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cdfcf700 1 -- 172.21.15.33:0/3543390806 learned_addr learned my addr 172.21.15.33:0/3543390806 (peer_addr_for_me v2:172.21.15.33:0/0) 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 -- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 msgr2=0x7f50c8067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 0x7f50c8067840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 -- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 msgr2=0x7f50c8153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c8153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 -- 172.21.15.33:0/3543390806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009420 con 0x7f50c80c5940 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cdfcf700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 0x7f50c8067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cd7ce700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T02:47:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50ce7d0700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 0x7f50c814ae70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f50c0001560 tx=0x7f50c0001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000a7e0 con 0x7f50c80c5940 2023-10-06T02:47:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f50c000a940 con 0x7f50c80c5940 2023-10-06T02:47:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c4009420 con 0x7f50c80c5940 2023-10-06T02:47:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000ad20 con 0x7f50c80c5940 2023-10-06T02:47:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c8068100 con 0x7f50c80c5940 2023-10-06T02:47:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.464+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c8071040 con 0x7f50c80c5940 2023-10-06T02:47:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.468+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f50c0007300 con 0x7f50c80c5940 2023-10-06T02:47:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.468+0000 7f50bdffb700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f50ac060580 0x7f50ac062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T02:47:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.468+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5612+0+0 (secure 0 0 0) 0x7f50c007c9f0 con 0x7f50c80c5940 2023-10-06T02:47:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.468+0000 7f50cdfcf700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f50ac060580 0x7f50ac062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T02:47:31.474 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.468+0000 7f50cdfcf700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f50ac060580 0x7f50ac062a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006d80 tx=0x7f50c40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T02:47:31.480 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.476+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f50c004d020 con 0x7f50c80c5940 2023-10-06T02:47:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.676+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50c806a1c0 con 0x7f50c80c5940 2023-10-06T02:47:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.676+0000 7f50bdffb700 1 -- 172.21.15.33:0/3543390806 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f50c0001aa0 con 0x7f50c80c5940 2023-10-06T02:47:31.682 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-10-06T02:47:31.683 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4267440543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4267440543},{"type":"v1","addr":"172.21.15.33:6833","nonce":4267440543}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3461907117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3461907117},{"type":"v1","addr":"172.21.15.167:6835","nonce":3461907117}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/4152072568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4152072568},{"type":"v1","addr":"172.21.15.167:6837","nonce":4152072568}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3199932646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3199932646},{"type":"v1","addr":"172.21.15.33:6835","nonce":3199932646}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-10-06T02:47:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f50ac060580 msgr2=0x7f50ac062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f50ac060580 0x7f50ac062a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006d80 tx=0x7f50c40091a0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 msgr2=0x7f50c814ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 0x7f50c814ae70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f50c0001560 tx=0x7f50c0001960 comp rx=0 tx=0).stop 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 shutdown_connections 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c80c5940 0x7f50c814ae70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50c80ade00 0x7f50c8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:6832/33311,v1:172.21.15.167:6833/33311] conn(0x7f50ac060580 0x7f50ac062a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 --2- 172.21.15.33:0/3543390806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50c8066fb0 0x7f50c8067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 >> 172.21.15.33:0/3543390806 conn(0x7f50c80c8240 msgr2=0x7f50c80693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 shutdown_connections 2023-10-06T02:47:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2023-10-06T02:47:31.680+0000 7f50cefd1700 1 -- 172.21.15.33:0/3543390806 wait complete. 2023-10-06T02:47:31.687 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 18 2023-10-06T02:47:31.697 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T02:47:31.697 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T02:47:31.697 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T02:47:31.698 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:31.711 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T02:47:31.711 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T02:47:31.712 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:31.723 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T02:47:31.723 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T02:47:31.723 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:31.740 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T02:47:31.740 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T02:47:31.740 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:31.797 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T02:47:31.797 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T02:47:31.797 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T02:47:31.797 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:31.857 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T02:47:31.858 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T02:47:31.858 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:31.936 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T02:47:31.937 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T02:47:31.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.016 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T02:47:32.016 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T02:47:32.016 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.078 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T02:47:32.079 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T02:47:32.079 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.144 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T02:47:32.144 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T02:47:32.144 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.214 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T02:47:32.214 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T02:47:32.214 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.283 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T02:47:32.283 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T02:47:32.283 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.355 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T02:47:32.355 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T02:47:32.355 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T02:47:32.355 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.380 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T02:47:32.381 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T02:47:32.381 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-10-06T02:47:32.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-10-06T02:47:32.408 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T02:47:32.408 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T02:47:32.408 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T02:47:32.409 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.442 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T02:47:32.443 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T02:47:32.443 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.461 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T02:47:32.461 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T02:47:32.461 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T02:47:32.522 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T02:47:32.522 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T02:47:32.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T02:47:32.546 INFO:teuthology.orchestra.run.smithi033.stdout:1696560312.712963 mon.a (mon.0) 112 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T02:47:32.546 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T02:47:32.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T02:47:32.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T02:47:32.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T02:47:32.643 INFO:teuthology.orchestra.run.smithi033.stdout:1696560312.712963 mon.a (mon.0) 112 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T02:47:32.643 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:47:32.643 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T02:47:32.983 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:47:32.983 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T02:47:33.331 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:47:33.331 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T02:47:33.499 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:47:33.500 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T02:47:33.672 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:47:33.673 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T02:47:33.907 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:47:33.908 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T02:47:34.131 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:47:34.132 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T02:47:34.343 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:47:34.343 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T02:47:34.539 INFO:tasks.ceph:Archiving mon data... 2023-10-06T02:47:34.540 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/data/mon.a.tgz 2023-10-06T02:47:34.540 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2023-10-06T02:47:34.547 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.WNsFIpYwfK 2023-10-06T02:47:34.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WNsFIpYwfK 2023-10-06T02:47:34.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.WNsFIpYwfK 2023-10-06T02:47:34.653 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.WNsFIpYwfK is 296KB 2023-10-06T02:47:34.718 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.WNsFIpYwfK 2023-10-06T02:47:34.726 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/data/mon.b.tgz 2023-10-06T02:47:34.726 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T02:47:34.733 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.gkgl4dhQaK 2023-10-06T02:47:34.733 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gkgl4dhQaK 2023-10-06T02:47:34.871 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.gkgl4dhQaK 2023-10-06T02:47:34.898 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.gkgl4dhQaK is 393KB 2023-10-06T02:47:34.959 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.gkgl4dhQaK 2023-10-06T02:47:34.965 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/data/mon.c.tgz 2023-10-06T02:47:34.965 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T02:47:34.972 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.P7DLiD5p6E 2023-10-06T02:47:34.972 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.P7DLiD5p6E 2023-10-06T02:47:35.067 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.P7DLiD5p6E 2023-10-06T02:47:35.084 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.P7DLiD5p6E is 393KB 2023-10-06T02:47:35.148 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.P7DLiD5p6E 2023-10-06T02:47:35.155 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T02:47:35.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T02:47:35.158 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T02:47:35.172 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T02:47:35.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T02:47:35.213 DEBUG:teuthology.orchestra.run.smithi167:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T02:47:35.240 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T02:47:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T02:47:35.252 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T02:47:35.254 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/remote/smithi033/crash 2023-10-06T02:47:35.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T02:47:35.272 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/remote/smithi167/crash 2023-10-06T02:47:35.274 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T02:47:35.291 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T02:47:35.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:47:35.294 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T02:47:35.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T02:47:35.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T02:47:35.422 INFO:tasks.ceph:Compressing logs... 2023-10-06T02:47:35.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:47:35.425 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:47:37.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-10-06T02:47:37.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-10-06T02:47:38.979 INFO:tasks.ceph:Archiving logs... 2023-10-06T02:47:38.979 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/remote/smithi033/log 2023-10-06T02:47:38.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T02:47:39.570 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/remote/smithi167/log 2023-10-06T02:47:39.571 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T02:47:39.991 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T02:47:40.000 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T02:47:40.000 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T02:47:40.003 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T02:47:40.085 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T02:47:40.095 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T02:47:40.095 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:47:40.098 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:47:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T02:47:40.120 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2023-10-06T02:47:40.120 INFO:teuthology.orchestra.run.smithi033.stdout:*hv01.front.sepi 44.4.53.2 2 u 41 64 77 0.118 0.090 1.586 2023-10-06T02:47:40.121 INFO:teuthology.orchestra.run.smithi033.stdout:+hv02.front.sepi 63.231.80.2 3 u 40 64 77 0.060 1.063 0.899 2023-10-06T02:47:40.121 INFO:teuthology.orchestra.run.smithi033.stdout:+hv03.front.sepi 74.6.168.72 3 u 37 64 77 0.086 -0.232 1.001 2023-10-06T02:47:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:47:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T02:47:40.123 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2023-10-06T02:47:40.123 INFO:teuthology.orchestra.run.smithi167.stdout:*hv01.front.sepi 44.4.53.2 2 u 42 64 77 0.078 1.539 1.037 2023-10-06T02:47:40.123 INFO:teuthology.orchestra.run.smithi167.stdout:+hv02.front.sepi 63.231.80.2 3 u 37 64 77 0.083 0.811 0.334 2023-10-06T02:47:40.123 INFO:teuthology.orchestra.run.smithi167.stdout:+hv03.front.sepi 74.6.168.72 3 u 38 64 77 0.083 -0.595 0.572 2023-10-06T02:47:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T02:47:40.124 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T02:47:40.133 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T02:47:40.134 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T02:47:40.149 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T02:47:40.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T02:47:40.183 INFO:teuthology.task.internal:Duration was 718.865651 seconds 2023-10-06T02:47:40.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T02:47:40.199 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T02:47:40.199 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T02:47:40.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T02:47:40.255 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T02:47:40.255 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2023-10-06T02:47:40.256 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T02:47:40.266 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T02:47:40.267 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T02:47:40.277 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T02:47:40.277 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:47:40.279 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T02:47:40.311 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T02:47:40.311 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T02:47:40.353 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T02:47:40.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T02:47:40.439 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T02:47:40.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T02:47:40.455 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T02:47:40.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T02:47:40.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:47:40.483 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:47:40.495 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2023-10-06T02:47:40.527 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2023-10-06T02:47:40.543 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:47:40.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:47:40.557 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T02:47:40.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:47:40.563 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T02:47:40.577 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T02:47:40.578 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/remote/smithi033 2023-10-06T02:47:40.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T02:47:40.603 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156/remote/smithi167 2023-10-06T02:47:40.603 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T02:47:40.626 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T02:47:40.626 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T02:47:40.641 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T02:47:40.670 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T02:47:40.686 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T02:47:40.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T02:47:40.703 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T02:47:40.704 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T02:47:40.706 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T02:47:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 02:47 /home/ubuntu/cephtest 2023-10-06T02:47:40.713 INFO:teuthology.orchestra.run.smithi167.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 02:47 /home/ubuntu/cephtest 2023-10-06T02:47:40.716 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T02:47:40.731 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T02:47:40.833 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T02:47:40.861 DEBUG:teuthology.nuke:shortname: smithi033 2023-10-06T02:47:40.861 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:47:40.884 DEBUG:teuthology.nuke:shortname: smithi167 2023-10-06T02:47:40.884 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:47:40.887 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:26:16.714597', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-06T02:47:40.914 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:26:16.713100', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-06T02:47:40.941 INFO:teuthology.orchestra.console.smithi033:Power off 2023-10-06T02:47:40.942 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T02:47:40.967 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2023-10-06T02:47:40.967 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:40.984 INFO:teuthology.orchestra.console.smithi167:Power off 2023-10-06T02:47:40.984 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T02:47:40.993 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-10-06T02:47:41.006 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-10-06T02:47:41.006 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:41.029 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T02:47:44.994 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:45.030 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:45.105 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-10-06T02:47:45.139 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T02:47:49.106 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:49.139 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:49.218 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2023-10-06T02:47:49.252 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-10-06T02:47:49.352 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-10-06T02:47:53.220 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T02:47:53.329 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2023-10-06T02:47:53.430 INFO:teuthology.orchestra.console.smithi033:Power off completed 2023-10-06T02:47:53.534 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} duration: 718.8656506538391 failure_reason: '"1696560312.712963 mon.a (mon.0) 112 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T02:47:53.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T02:47:53.617 INFO:teuthology.run:FAIL